{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vZ9z9-sX60Ct"
   },
   "source": [
    "# Parallel is All You Want: Combining Spatial and Temporal Feature Representions of Speech Emotion by Parallelizing CNNs and Transformer-Encoders\n",
    "# Abstract\n",
    "In this notebook, I'm going to build upon my [Intro to Speech Audio Classification repo](https://github.com/IliaZenkov/sklearn-audio-classification) and build two parallel convolutional neural networks (CNN) in parallel with a Transformer encoder network to classify audio data. We're working on the [RAVDESS dataset](https://smartlaboratory.org/ravdess/) to classify emotions from one of 8 classes. We combine the CNN for spatial feature representation and the Transformer for temporal feature representation. We augment the training data by increasing variation in the dataset to reduce overfitting; we use Additive White Gaussian Noise (AWGN) to augment the RAVDESS dataset three-fold for a total of 4320 audio samples.\n",
    "\n",
    "We harness the image-classification and spatial feature representation power of the CNN by treating MFCC plots as grayscale images; their width is a time scale, their height is a frequency scale. The value of each pixel in the MFCC is the intensity of the audio signal at a particular range of mel frequencies at a time step. \n",
    "\n",
    "Because of the sequential nature of the data, we will also use the Transformer to try and model as accurately as possible the temporal relationships between pitch transitions in emotions.  \n",
    "\n",
    "This notebook takes inspirations from a variety of recent advances in deep learning and network architectures; in particular, stacked and parallel CNN networks combined with multi-head self-attention layers from the Transformer Encoder. I hypothesize that the expansion of CNN filter channel dimensions and reduction of feature maps will provide the most expressive feature representation at the lowest computational cost, while the Transformer-Encoder is used with the hypothesis that the network will learn to predict frequency distributions of different emotions according to the global structure of the MFCC plot (and indirectly, mel spectrogram) of each emotion. **With the strength of the CNN in spatial feature representation and Transformer in sequence encoding, I manage to achieve an 80.44% accuracy on a hold-out test set from the RAVDESS dataset.**\n",
    "\n",
    "<!--TABLE OF CONTENTS-->\n",
    "# Table of Contents\n",
    "- [Introduction](#Introduction)\n",
    "  - [Define Features](#Define-features)\n",
    "  - [Load Data and Extract Features](#Load-Data-and-Extract-Features)\n",
    "  - [Split into Train/Validation/Test Sets](#Split-into-Train/Validation/Test-Sets)\n",
    "  - [Augmenting the Data with AWGN: Additive White Gaussian Noise](#Augmenting-the-Data-with-AWGN---Additive-White-Gaussian-Noise)\n",
    "  - [Format Data into Tensor-Ready 4D Arrays](#Format-Data-into-Tensor-Ready-4D-Arrays)\n",
    "  - [Feature Scaling](#Feature-Scaling)\n",
    "  - [Save and Reload Data as NumPy Arrays](#Save-and-Reload-Data-as-NumPy-Arrays)\n",
    "- [Architecture Overview](#Architecture-Overview)\n",
    "- [CNN Motivation](#CNN-Motivation)\n",
    "- [Transformer-Encoder Motivation](#Transformer-Encoder-Motivation)\n",
    "- [Building Model Architecture and Forward Pass](#Build-Model-Architecture-and-Define-Forward-Pass)\n",
    "  - [Analyzing The Flow of Tensors Through the Network](#Analyzing-The-Flow-of-Tensors-Through-the-Network)\n",
    "  - [Choosing Loss Function](#Define-Loss/Criterion)\n",
    "  - [Choosing Optimizer](#Choose-Optimizer)\n",
    "  - [Build Training Step](#Define-Training-Step)\n",
    "  - [Build Validation Step](#Define-Validation-Step)\n",
    "  - [Make Checkpoint Functions](#Make-Checkpoint-Functions)\n",
    "- [Build Training Loop](#Build-Training-Loop)\n",
    "  - [Train Model](#Train-Model)\n",
    "- [Check the Loss Curve's Behaviour](#Check-the-Loss-Curve's-Behaviour)\n",
    "- [Evaluate Performance on Test Set](#Evaluate-the-Model-on-Hold-Out-Test-Set)\n",
    "- [Conclusion](#Conclusion)\n",
    "- [References](#References)\n",
    "\n",
    "# Appendices\n",
    "- [Appendix A: Convolutional Neural Nets Dissected](#Appendix-A---Convolutional-Neural-Nets-Dissected)\n",
    "  - [Kernels and Filters](#Kernels-and-Filters)\n",
    "  - [Zero Padding](#Zero-Padding)\n",
    "  - [Max Pooling](#Max-Pooling)\n",
    "  - [Regularization Using Dropout and a note on Pruning](#Regularization-Using-Dropout-and-a-note-on-Pruning)\n",
    "  - [Batch Normalization and Optimizing the Optimization Landsape](#Batch-Normalization-and-Optimizing-the-Optimization-Landsape)\n",
    "  - [ReLU: Non-Saturated Activations are Healthier (for CNNs)](#ReLU---Non-Saturated-Activations-are-Healthier-[for-CNNs])\n",
    "  - [Turning Feature Maps into Probabilities with a Fully Connected Layer](#Turning-Feature-Maps-into-Probabilities-with-a-Fully-Connected-Layer)\n",
    "- [Appendix B: The Transformer](#Appendix-B---The-Transformer)\n",
    "    - [Self-Attention](#Self-Attention)\n",
    "    - [Multi-Head Self-Attention](#Multi-Head-Self-Attention)\n",
    "    - [Transformer Architecture](#Transformer-Architecture)\n",
    "        - [Input](#Input)\n",
    "        - [Encoder](#Encoder)\n",
    "        - [Decoder](#Decoder)\n",
    "        - [Output](#Output)\n",
    "- [Appendix C: From Autoencoders to LSTMs to Attention](#Appendix-C---From-Autoencoders-to-LSTMs-to-Attention)\n",
    "  - [Autoencoders](#Autoencoders)\n",
    "  - [The Sparse Autoencoder](#The-Sparse-Autoencoder)\n",
    "  - [The Variational Autoencoder, Decoders, and Latent Space](#The-Variational-Autoencoder,-Decoders,-and-Latent-Space)\n",
    "  - [The RNN, Seq2Seq, and Gradient Problems](#The-RNN,-Seq2Seq,-and-Gradient-Problems)\n",
    "  - [The LSTM Cell](#The-LSTM-Cell)\n",
    "  - [Bidirectional RNN](#Bidirectional-RNN)\n",
    "  - [The Attention Mechanism](#The-Attention-Mechanism)\n",
    "- [Appendix D: Supplementary Notes](#Appendix-D---Supplementary-Notes)\n",
    "  - [More CNN Kernel/Filter Math](#More-CNN-Kernel/Filter-Math)\n",
    "  - [Smoothing the Optimization Surface](#Smoothing-the-Optimization-Surface)\n",
    "\n",
    "\n",
    "# Introduction\n",
    "From my previous notebook: \"Long-Short-Term-Memory Recurrent Neural Networks (LSTM RNNs) and Convolutional Neural Networks (CNNs) are excellent DNN candidates for audio data classification: LSTM RNNs because of their excellent ability to interpret sequential data such as features of the audio waveform represented as a time series; CNNs because features engineered on audio data such as spectrograms have marked resemblance to images, in which CNNs excel at recognizing and discriminating between distinct patterns.\" - Me \n",
    "\n",
    "I'm going to build on that - CNNs are still the hallmark of image classification today, although even in this domain Transformers are beginning to take the main stage: A [2021 ICLR submission: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://openreview.net/forum?id=YicbFdNTTy) claims they've implemented a Transformer for image classification that outperforms a state of the art CNN, and at a much lower computational complexity.\n",
    "\n",
    "In addition to taking inspiration from the above, it's also no longer 2015 - so instead of the LSTM-RNN I'm going to implement its successor the Transformer model in parallel with a CNN to try and get state-of-the-art performance on the RAVDESS dataset. \n",
    "\n",
    "Other motivations for the architecture of this model come from a variety of papers from the past few years. **The most notable inspirations are:**\n",
    "- The Transformer: [Attention is All You Need](https://arxiv.org/abs/1706.03762) for the Transformer\n",
    "- Inception and GoogLeNet: [Going Deeper with Convolutions](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43022.pdf) for parallel, stacked CNNs\n",
    "- AlexNet: [ImageNet Classification with Deep Convolutional\n",
    "](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) for increasing the complexity of feature maps with deeper CNN networks, as well as data augmentation by adding modified versions of the training data to itself\n",
    "- VGGNet: [Very Deep Convolutional Networks For Large-Scale Image Recognition](https://arxiv.org/pdf/1409.1556.pdf) for using fixed size kernels throughout stacked CNN layers\n",
    "- LeNet: [Gradient-Based Learning Applied to Document Recognition](http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf) for the convolution>pool>convolution>pool paradigm\n",
    "- Self-Attention: [Long Short-Term Memory-Networks for Machine Reading](https://arxiv.org/pdf/1601.06733.pdf) for understanding Transformer architecture\n",
    "- Dropout regularization: [Dropout: A Simple Way to Prevent Neural Networks from\n",
    "Overfitting](https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf) speaks for itself\n",
    "- Batch Norm: [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://arxiv.org/abs/1502.03167) speaks for itself\n",
    "\n",
    "Let's get to it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ExkoPoNs60Cu"
   },
   "source": [
    "#### Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "id": "ZcoBEft660Cv"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import os, glob\n",
    "import librosa\n",
    "import librosa.display\n",
    "import IPython\n",
    "from IPython.display import Audio\n",
    "from IPython.display import Image\n",
    "import warnings; warnings.filterwarnings('ignore') #matplot lib complains about librosa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "38YJn86i60Cz"
   },
   "outputs": [],
   "source": [
    "#google colab has an old version of librosa with missing mel spectrogram args (for MFCC); upgrade to current\n",
    "!pip install -U librosa "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "wXgX4hsZ60C1",
    "outputId": "69b4373d-455e-4a4b-fb26-0f2d33209a1e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mounted at /content/gdrive\n"
     ]
    }
   ],
   "source": [
    "# needed to import dataset from google drive into colab \n",
    "from google.colab import drive\n",
    "drive.mount(\"/content/gdrive\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "T8GiBMh2CJCR"
   },
   "outputs": [],
   "source": [
    "# copy RAVDESS dataset from gdrive and unzip\n",
    "!cp '/content/gdrive/My Drive/DL/RAVDESS.zip' .\n",
    "!unzip -q RAVDESS.zip "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ADgTrt0X60C3"
   },
   "source": [
    "## Define features\n",
    "\n",
    "Define features as in the previous notebook on this task from my ['sklearn-audio-classification' repo](https://github.com/IliaZenkov/sklearn-audio-classification). That notebook explains the motivation behind the Mel Spectrogram and its derivative MFCC, which we use as a feature. In short, we're looking for transitions in audible pitch frequencies. \n",
    "\n",
    "**MFCCs alone provide the best accuracy in this model with training considerations in mind - and provide as good an accuracy as using chromagrams + mel spectrograms + MFCCs. We don't want extra complexity in a highly parameterized deep neural net such as this one** (unless we absolutely need it).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "id": "OBZMdi7l60C4"
   },
   "outputs": [],
   "source": [
    "# RAVDESS native sample rate is 48k\n",
    "sample_rate = 48000\n",
    "\n",
    "# Mel Spectrograms are not directly used as a feature in this model\n",
    "# Mel Spectrograms are used in calculating MFCCs, which are a higher-level representation of pitch transition\n",
    "# MFCCs work better - left the mel spectrogram function here in case anyone wants to experiment\n",
    "def feature_melspectrogram(\n",
    "    waveform, \n",
    "    sample_rate,\n",
    "    fft = 1024,\n",
    "    winlen = 512,\n",
    "    window='hamming',\n",
    "    hop=256,\n",
    "    mels=128,\n",
    "    ):\n",
    "    \n",
    "    # Produce the mel spectrogram for all STFT frames and get the mean of each column of the resulting matrix to create a feature array\n",
    "    # Using 8khz as upper frequency bound should be enough for most speech classification tasks\n",
    "    melspectrogram = librosa.feature.melspectrogram(\n",
    "        y=waveform, \n",
    "        sr=sample_rate, \n",
    "        n_fft=fft, \n",
    "        win_length=winlen, \n",
    "        window=window, \n",
    "        hop_length=hop, \n",
    "        n_mels=mels, \n",
    "        fmax=sample_rate/2)\n",
    "    \n",
    "    # convert from power (amplitude**2) to decibels\n",
    "    # necessary for network to learn - doesn't converge with raw power spectrograms \n",
    "    melspectrogram = librosa.power_to_db(melspectrogram, ref=np.max)\n",
    "    \n",
    "    return melspectrogram\n",
    "\n",
    "def feature_mfcc(\n",
    "    waveform, \n",
    "    sample_rate,\n",
    "    n_mfcc = 40,\n",
    "    fft = 1024,\n",
    "    winlen = 512,\n",
    "    window='hamming',\n",
    "    #hop=256, # increases # of time steps; was not helpful\n",
    "    mels=128\n",
    "    ):\n",
    "\n",
    "    # Compute the MFCCs for all STFT frames \n",
    "    # 40 mel filterbanks (n_mfcc) = 40 coefficients\n",
    "    mfc_coefficients=librosa.feature.mfcc(\n",
    "        y=waveform, \n",
    "        sr=sample_rate, \n",
    "        n_mfcc=n_mfcc,\n",
    "        n_fft=fft, \n",
    "        win_length=winlen, \n",
    "        window=window, \n",
    "        #hop_length=hop, \n",
    "        n_mels=mels, \n",
    "        fmax=sample_rate/2\n",
    "        ) \n",
    "\n",
    "    return mfc_coefficients\n",
    "\n",
    "def get_features(waveforms, features, samplerate):\n",
    "\n",
    "    # initialize counter to track progress\n",
    "    file_count = 0\n",
    "\n",
    "    # process each waveform individually to get its MFCCs\n",
    "    for waveform in waveforms:\n",
    "        mfccs = feature_mfcc(waveform, sample_rate)\n",
    "        features.append(mfccs)\n",
    "        file_count += 1\n",
    "        # print progress \n",
    "        print('\\r'+f' Processed {file_count}/{len(waveforms)} waveforms',end='')\n",
    "    \n",
    "    # return all features from list of waveforms\n",
    "    return features\n",
    "\n",
    "def get_waveforms(file):\n",
    "    \n",
    "    # load an individual sample audio file\n",
    "    # read the full 3 seconds of the file, cut off the first 0.5s of silence; native sample rate = 48k\n",
    "    # don't need to store the sample rate that librosa.load returns\n",
    "    waveform, _ = librosa.load(file, duration=3, offset=0.5, sr=sample_rate)\n",
    "    \n",
    "    # make sure waveform vectors are homogenous by defining explicitly\n",
    "    waveform_homo = np.zeros((int(sample_rate*3,)))\n",
    "    waveform_homo[:len(waveform)] = waveform\n",
    "    \n",
    "    # return a single file's waveform                                      \n",
    "    return waveform_homo\n",
    "    \n",
    "# RAVDESS dataset emotions\n",
    "# shift emotions left to be 0 indexed for PyTorch\n",
    "emotions_dict ={\n",
    "    '0':'surprised',\n",
    "    '1':'neutral',\n",
    "    '2':'calm',\n",
    "    '3':'happy',\n",
    "    '4':'sad',\n",
    "    '5':'angry',\n",
    "    '6':'fearful',\n",
    "    '7':'disgust'\n",
    "}\n",
    "\n",
    "# Additional attributes from RAVDESS to play with\n",
    "emotion_attributes = {\n",
    "    '01': 'normal',\n",
    "    '02': 'strong'\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VoHM-Dh860C6"
   },
   "source": [
    "## Load Data and Extract Features\n",
    "\n",
    "\n",
    "We process each file in the dataset and extract its features.\n",
    "\n",
    "We return the waveforms and the labels (from the file names of the RAVDESS audio samples). We return the raw waveforms because we're going to do some extra processing.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "id": "XasJEOjN60C7"
   },
   "outputs": [],
   "source": [
    "# path to data for glob\n",
    "data_path = 'RAVDESS dataset/Actor_*/*.wav'\n",
    "\n",
    "def load_data():\n",
    "    # features and labels\n",
    "    emotions = []\n",
    "    # raw waveforms to augment later\n",
    "    waveforms = []\n",
    "    # extra labels\n",
    "    intensities, genders = [],[]\n",
    "    # progress counter\n",
    "    file_count = 0\n",
    "    for file in glob.glob(data_path):\n",
    "        # get file name with labels\n",
    "        file_name = os.path.basename(file)\n",
    "        \n",
    "        # get emotion label from the sample's file\n",
    "        emotion = int(file_name.split(\"-\")[2])\n",
    "\n",
    "        #  move surprise to 0 for cleaner behaviour with PyTorch/0-indexing\n",
    "        if emotion == 8: emotion = 0 # surprise is now at 0 index; other emotion indeces unchanged\n",
    "\n",
    "        # can convert emotion label to emotion string if desired, but\n",
    "        # training on number is better; better convert to emotion string after predictions are ready\n",
    "        # emotion = emotions_dict[str(emotion)]\n",
    "        \n",
    "        # get other labels we might want\n",
    "        intensity = emotion_attributes[file_name.split(\"-\")[3]]\n",
    "        # even actors are female, odd are male\n",
    "        if (int((file_name.split(\"-\")[6]).split(\".\")[0]))%2==0: \n",
    "            gender = 'female' \n",
    "        else: \n",
    "            gender = 'male'\n",
    "            \n",
    "        # get waveform from the sample\n",
    "        waveform = get_waveforms(file)\n",
    "        \n",
    "        # store waveforms and labels\n",
    "        waveforms.append(waveform)\n",
    "        emotions.append(emotion)\n",
    "        intensities.append(intensity) # store intensity in case we wish to predict\n",
    "        genders.append(gender) # store gender in case we wish to predict \n",
    "        \n",
    "        file_count += 1\n",
    "        # keep track of data loader's progress\n",
    "        print('\\r'+f' Processed {file_count}/{1440} audio samples',end='')\n",
    "        \n",
    "    return waveforms, emotions, intensities, genders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "37LrPb2i60C9",
    "outputId": "ae9084c0-1ec5-4f9a-f748-97a8faec2dc8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Processed 1440/1440 audio samples"
     ]
    }
   ],
   "source": [
    "# load data \n",
    "# init explicitly to prevent data leakage from past sessions, since load_data() appends\n",
    "waveforms, emotions, intensities, genders = [],[],[],[]\n",
    "waveforms, emotions, intensities, genders = load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PLBp0B-n60DB"
   },
   "source": [
    "## Check extracted audio waveforms and labels:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 67
    },
    "id": "pjjUpfNm60DC",
    "outputId": "e752e45d-6499-4040-cf0d-2129f8bf09f7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waveforms set: 1440 samples\n",
      "Waveform signal length: 144000\n",
      "Emotions set: 1440 sample labels\n"
     ]
    }
   ],
   "source": [
    "print(f'Waveforms set: {len(waveforms)} samples')\n",
    "# we have 1440 waveforms but we need to know their length too; should be 3 sec * 48k = 144k\n",
    "print(f'Waveform signal length: {len(waveforms[0])}')\n",
    "print(f'Emotions set: {len(emotions)} sample labels')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Zkfe5cWQGa-P"
   },
   "source": [
    "Looks good. 1440 samples and 1440 labels in total.\n",
    "\n",
    "**Waveforms are 144k long because 3 seconds * 48k sample rate = 144k length array representing the 3 second audio snippet.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UDnnCn8HEOQq"
   },
   "source": [
    "## Split into Train/Validation/Test Sets\n",
    "We'll use an 80/10/10 train/validation/test split to maximize training data and keep a reasonable validation/test set. \n",
    "\n",
    "**We're splitting waveforms so we can process train/validation/test waveforms separately and avoid data leakage.** \n",
    "\n",
    "**Have to take care to split the sets proportionally w.r.t. emotion.**\n",
    "\n",
    "**Yes, we can use sklearn - but to convince myself I've eradicated data leakage issues I did this manually.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 101
    },
    "id": "ofaAetu48YEg",
    "outputId": "9c453120-d1db-4d62-ac2a-02fed87289b7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training waveforms:(1147, 144000), y_train:(1147,)\n",
      "Validation waveforms:(143, 144000), y_valid:(143,)\n",
      "Test waveforms:(150, 144000), y_test:(150,)\n",
      "\n",
      "Sets are unique: 1440 samples out of 1440 are unique\n"
     ]
    }
   ],
   "source": [
    "# create storage for train, validation, test sets and their indices\n",
    "train_set,valid_set,test_set = [],[],[]\n",
    "X_train,X_valid,X_test = [],[],[]\n",
    "y_train,y_valid,y_test = [],[],[]\n",
    "\n",
    "# convert waveforms to array for processing\n",
    "waveforms = np.array(waveforms)\n",
    "\n",
    "# process each emotion separately to make sure we builf balanced train/valid/test sets \n",
    "for emotion_num in range(len(emotions_dict)):\n",
    "        \n",
    "    # find all indices of a single unique emotion\n",
    "    emotion_indices = [index for index, emotion in enumerate(emotions) if emotion==emotion_num]\n",
    "\n",
    "    # seed for reproducibility \n",
    "    np.random.seed(69)\n",
    "    # shuffle indicies \n",
    "    emotion_indices = np.random.permutation(emotion_indices)\n",
    "\n",
    "    # store dim (length) of the emotion list to make indices\n",
    "    dim = len(emotion_indices)\n",
    "\n",
    "    # store indices of training, validation and test sets in 80/10/10 proportion\n",
    "    # train set is first 80%\n",
    "    train_indices = emotion_indices[:int(0.8*dim)]\n",
    "    # validation set is next 10% (between 80% and 90%)\n",
    "    valid_indices = emotion_indices[int(0.8*dim):int(0.9*dim)]\n",
    "    # test set is last 10% (between 90% - end/100%)\n",
    "    test_indices = emotion_indices[int(0.9*dim):]\n",
    "\n",
    "    # create train waveforms/labels sets\n",
    "    X_train.append(waveforms[train_indices,:])\n",
    "    y_train.append(np.array([emotion_num]*len(train_indices),dtype=np.int32))\n",
    "    # create validation waveforms/labels sets\n",
    "    X_valid.append(waveforms[valid_indices,:])\n",
    "    y_valid.append(np.array([emotion_num]*len(valid_indices),dtype=np.int32))\n",
    "    # create test waveforms/labels sets\n",
    "    X_test.append(waveforms[test_indices,:])\n",
    "    y_test.append(np.array([emotion_num]*len(test_indices),dtype=np.int32))\n",
    "\n",
    "    # store indices for each emotion set to verify uniqueness between sets \n",
    "    train_set.append(train_indices)\n",
    "    valid_set.append(valid_indices)\n",
    "    test_set.append(test_indices)\n",
    "\n",
    "# concatenate, in order, all waveforms back into one array \n",
    "X_train = np.concatenate(X_train,axis=0)\n",
    "X_valid = np.concatenate(X_valid,axis=0)\n",
    "X_test = np.concatenate(X_test,axis=0)\n",
    "\n",
    "# concatenate, in order, all emotions back into one array \n",
    "y_train = np.concatenate(y_train,axis=0)\n",
    "y_valid = np.concatenate(y_valid,axis=0)\n",
    "y_test = np.concatenate(y_test,axis=0)\n",
    "\n",
    "# combine and store indices for all emotions' train, validation, test sets to verify uniqueness of sets\n",
    "train_set = np.concatenate(train_set,axis=0)\n",
    "valid_set = np.concatenate(valid_set,axis=0)\n",
    "test_set = np.concatenate(test_set,axis=0)\n",
    "\n",
    "# check shape of each set\n",
    "print(f'Training waveforms:{X_train.shape}, y_train:{y_train.shape}')\n",
    "print(f'Validation waveforms:{X_valid.shape}, y_valid:{y_valid.shape}')\n",
    "print(f'Test waveforms:{X_test.shape}, y_test:{y_test.shape}')\n",
    "\n",
    "# make sure train, validation, test sets have no overlap/are unique\n",
    "# get all unique indices across all sets and how many times each index appears (count)\n",
    "uniques, count = np.unique(np.concatenate([train_set,test_set,valid_set],axis=0), return_counts=True)\n",
    "\n",
    "# if each index appears just once, and we have 1440 such unique indices, then all sets are unique\n",
    "if sum(count==1) == len(emotions):\n",
    "    print(f'\\nSets are unique: {sum(count==1)} samples out of {len(emotions)} are unique')\n",
    "else:\n",
    "    print(f'\\nSets are NOT unique: {sum(count==1)} samples out of {len(emotions)} are unique')    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kikSzy0MCFix"
   },
   "source": [
    "## Extract Features \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HJ0phzMzy40S"
   },
   "source": [
    "Extract the features from unaugmented waveforms first. In the next step, we'll append features from augmented waveforms to these 'native' features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 202
    },
    "id": "OI4y4gFgCED3",
    "outputId": "12fad87e-9b6b-4b1a-d48c-f5d40d7e034a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train waveforms:\n",
      " Processed 1147/1147 waveforms\n",
      "\n",
      "Validation waveforms:\n",
      " Processed 143/143 waveforms\n",
      "\n",
      "Test waveforms:\n",
      " Processed 150/150 waveforms\n",
      "\n",
      "Features set: 1440 total, 1147 train, 143 validation, 150 test samples\n",
      "Features (MFC coefficient matrix) shape: 40 mel frequency coefficients x 282 time steps\n"
     ]
    }
   ],
   "source": [
    "# initialize feature arrays\n",
    "# We extract MFCC features from waveforms and store in respective 'features' array\n",
    "features_train, features_valid, features_test = [],[],[]\n",
    "\n",
    "print('Train waveforms:') # get training set features \n",
    "features_train = get_features(X_train, features_train, sample_rate)\n",
    "\n",
    "print('\\n\\nValidation waveforms:') # get validation set features\n",
    "features_valid = get_features(X_valid, features_valid, sample_rate)\n",
    "\n",
    "print('\\n\\nTest waveforms:') # get test set features \n",
    "features_test = get_features(X_test, features_test, sample_rate)\n",
    "\n",
    "print(f'\\n\\nFeatures set: {len(features_train)+len(features_test)+len(features_valid)} total, {len(features_train)} train, {len(features_valid)} validation, {len(features_test)} test samples')\n",
    "print(f'Features (MFC coefficient matrix) shape: {len(features_train[0])} mel frequency coefficients x {len(features_train[0][1])} time steps')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mx3PipOL60DG"
   },
   "source": [
    "## Augmenting the Data with AWGN - Additive White Gaussian Noise\n",
    "\n",
    "### Motivation\n",
    "\n",
    "Since our dataset is small, it is prone to overfitting - especially with highly parameterized deep neural net models\n",
    "such as the one we aim to build in this notebook. As such, we're going to want to augment our data. Generating more real samples will be immensely difficult. Instead, we can add white noise to the audio signals - not only to mask the effect of random noise present in the training set - but also **to create pseudo-new training samples and offset the impact of noise intrinsic to the dataset.** \n",
    "\n",
    "In addition, the RAVDESS dataset is extremely clean - we will likely want to make predictions on noisy, real-world data - yet another reason to augment the training data.\n",
    "\n",
    "We're going to use Additive White Gaussian Noise (AWGN). It's Additive because we're adding it to the source audio signal,\n",
    "**it's Gaussian because the noise vector will be sampled from a normal distribution and have a time average of zero (zero-mean), and it's white because after a whitening transformation the noise will add power to the audio signal uniformly across the frequency distribution.**\n",
    "\n",
    "We need a good balance of noise - too little will be useless, and too much will make it too difficult for the network to learn from the training data. **Note that this is just for training - we would _not_ need to add AWGN to real-world data on which we make predictions** (although we could). \n",
    "\n",
    "### Math\n",
    "The key parameters in AWGN are the signal to noise ratio (SNR), defining the magnitude of the noise added w.r.t. the audio signal. We parameterize AWGN with the minimum and maximize SNR so we can pick a random SNR to use in augmenting each sample's waveform.\n",
    "\n",
    "We need to constrain covariance to make it true AWGN. **We make a zero-mean vector of Gaussian noises (np.random.normal) that are statistically dependent. We need to apply a [whitening transformation](https://en.wikipedia.org/wiki/Whitening_transformation)**, a linear transformation taking a vector of random normal (Gaussian) variables with a known covariance matrix and mapping it to a new vector whose covariance is the identity matrix, i.e. the vector is now perfectly uncorrelated with a diaganol covariance matrix, each point of noise having variance == stdev == 1. **The whitening transformation by definition transforms a vector into a white noise vector.**\n",
    "\n",
    "We're going to add the AWGN augmented waveforms as new samples to our dataset. **Since we generate AWGN which is random for each and every sample - random random noise - we can add multiples of our noise-augmented dataset. I'll add 2 extra identical, randomly noisy datasets with 1440 samples each to get a dataset with 1440 native + 1440x2 == 4320 noisy samples.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "T6iY69E-60DG"
   },
   "outputs": [],
   "source": [
    "def awgn_augmentation(waveform, multiples=2, bits=16, snr_min=15, snr_max=30): \n",
    "    \n",
    "    # get length of waveform (should be 3*48k = 144k)\n",
    "    wave_len = len(waveform)\n",
    "    \n",
    "    # Generate normally distributed (Gaussian) noises\n",
    "    # one for each waveform and multiple (i.e. wave_len*multiples noises)\n",
    "    noise = np.random.normal(size=(multiples, wave_len))\n",
    "    \n",
    "    # Normalize waveform and noise\n",
    "    norm_constant = 2.0**(bits-1)\n",
    "    norm_wave = waveform / norm_constant\n",
    "    norm_noise = noise / norm_constant\n",
    "    \n",
    "    # Compute power of waveform and power of noise\n",
    "    signal_power = np.sum(norm_wave ** 2) / wave_len\n",
    "    noise_power = np.sum(norm_noise ** 2, axis=1) / wave_len\n",
    "    \n",
    "    # Choose random SNR in decibels in range [15,30]\n",
    "    snr = np.random.randint(snr_min, snr_max)\n",
    "    \n",
    "    # Apply whitening transformation: make the Gaussian noise into Gaussian white noise\n",
    "    # Compute the covariance matrix used to whiten each noise \n",
    "    # actual SNR = signal/noise (power)\n",
    "    # actual noise power = 10**(-snr/10)\n",
    "    covariance = np.sqrt((signal_power / noise_power) * 10 ** (- snr / 10))\n",
    "    # Get covariance matrix with dim: (144000, 2) so we can transform 2 noises: dim (2, 144000)\n",
    "    covariance = np.ones((wave_len, multiples)) * covariance\n",
    "\n",
    "    # Since covariance and noise are arrays, * is the haddamard product \n",
    "    # Take Haddamard product of covariance and noise to generate white noise\n",
    "    multiple_augmented_waveforms = waveform + covariance.T * noise\n",
    "    \n",
    "    return multiple_augmented_waveforms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "tx-eNKa060DJ"
   },
   "outputs": [],
   "source": [
    "def augment_waveforms(waveforms, features, emotions, multiples):\n",
    "    # keep track of how many waveforms we've processed so we can add correct emotion label in the same order\n",
    "    emotion_count = 0\n",
    "    # keep track of how many augmented samples we've added\n",
    "    added_count = 0\n",
    "    # convert emotion array to list for more efficient appending\n",
    "    emotions = emotions.tolist()\n",
    "\n",
    "    for waveform in waveforms:\n",
    "\n",
    "        # Generate 2 augmented multiples of the dataset, i.e. 1440 native + 1440*2 noisy = 4320 samples total\n",
    "        augmented_waveforms = awgn_augmentation(waveform, multiples=multiples)\n",
    "\n",
    "        # compute spectrogram for each of 2 augmented waveforms\n",
    "        for augmented_waveform in augmented_waveforms:\n",
    "\n",
    "            # Compute MFCCs over augmented waveforms\n",
    "            augmented_mfcc = feature_mfcc(augmented_waveform, sample_rate=sample_rate)\n",
    "\n",
    "            # append the augmented spectrogram to the rest of the native data\n",
    "            features.append(augmented_mfcc)\n",
    "            emotions.append(emotions[emotion_count])\n",
    "\n",
    "            # keep track of new augmented samples\n",
    "            added_count += 1\n",
    "\n",
    "            # check progress\n",
    "            print('\\r'+f'Processed {emotion_count + 1}/{len(waveforms)} waveforms for {added_count}/{len(waveforms)*multiples} new augmented samples',end='')\n",
    "\n",
    "        # keep track of the emotion labels to append in order\n",
    "        emotion_count += 1\n",
    "        \n",
    "        # store augmented waveforms to check their shape\n",
    "        augmented_waveforms_temp.append(augmented_waveforms)\n",
    "    \n",
    "    return features, emotions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4GZ2CAy3F-nK"
   },
   "source": [
    "### Compute AWGN-augmented features and add to the rest of the dataset\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 218
    },
    "id": "05bOxnra54ia",
    "outputId": "c9decf24-7cab-4688-9836-06fede52ebba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train waveforms:\n",
      "Processed 1147/1147 waveforms for 2294/2294 new augmented samples\n",
      "\n",
      "Validation waveforms:\n",
      "Processed 143/143 waveforms for 286/286 new augmented samples\n",
      "\n",
      "Test waveforms:\n",
      "Processed 150/150 waveforms for 300/300 new augmented samples\n",
      "\n",
      "Native + Augmented Features set: 4320 total, 3441 train, 429 validation, 450 test samples\n",
      "3441 training sample labels, 429 validation sample labels, 450 test sample labels\n",
      "Features (MFCC matrix) shape: 40 mel frequency coefficients x 282 time steps\n"
     ]
    }
   ],
   "source": [
    "# store augmented waveforms to verify their shape and random-ness\n",
    "augmented_waveforms_temp = []\n",
    "\n",
    "# specify multiples of our dataset to add as augmented data\n",
    "multiples = 2\n",
    "\n",
    "print('Train waveforms:') # augment waveforms of training set\n",
    "features_train , y_train = augment_waveforms(X_train, features_train, y_train, multiples)\n",
    "\n",
    "print('\\n\\nValidation waveforms:') # augment waveforms of validation set\n",
    "features_valid, y_valid = augment_waveforms(X_valid, features_valid, y_valid, multiples)\n",
    "\n",
    "print('\\n\\nTest waveforms:') # augment waveforms of test set \n",
    "features_test, y_test = augment_waveforms(X_test, features_test, y_test, multiples)\n",
    "\n",
    "# Check new shape of extracted features and data:\n",
    "print(f'\\n\\nNative + Augmented Features set: {len(features_train)+len(features_test)+len(features_valid)} total, {len(features_train)} train, {len(features_valid)} validation, {len(features_test)} test samples')\n",
    "print(f'{len(y_train)} training sample labels, {len(y_valid)} validation sample labels, {len(y_test)} test sample labels')\n",
    "print(f'Features (MFCC matrix) shape: {len(features_train[0])} mel frequency coefficients x {len(features_train[0][1])} time steps')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pG6UTrZ160DO"
   },
   "source": [
    "### Check Augmented Waveforms:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 295
    },
    "id": "co938icT60DP",
    "outputId": "1a2e1335-4cf0-41f9-a7fe-93ebb896183c"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3kAAAEWCAYAAADbzV5zAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABuaElEQVR4nO3dd5xU1fk/8M8zsw2Wpfe6oCgiCAKhiF1Uioq9xPYzxRJNNNEk5GvXqKTYa2yJJYk1USIoimJDlCYISFuKsLQFFrawdWbO74+Z2b07O+XeuW3K5/16oVNuOXdn9555TnmOKKVAREREREREmcHjdgGIiIiIiIjIOgzyiIiIiIiIMgiDPCIiIiIiogzCII+IiIiIiCiDMMgjIiIiIiLKIAzyiIiIiIiIMgiDPKI0IyLvi8iVbpeDiIiI7CMin4rIz9wuB6UnBnlENhORLSKyW0QKNa/9TEQ+1bHvXSLyqvY1pdQUpdRLNhSViIiyQCh42C8i+ZrX1onIhZrnE0VERXmtWkRyQs97ichzIrIj9PomEfmHiAwJvV8cOsbsiPO/KiJ3JSjjiaF9f2fRZTtKc+05bpeFshODPCJn5AC40e1CEBFRdhORYgDHAVAAztK89TmAEzTPjwewNsprXymlfCLSBcBXANqGjlcEYBSAzwCcGnHa8SIy0WBRrwRQHvo/ERnEII/IGX8BcIuIdIx8Q0QeFZFtIlIpIktF5LjQ65MB/B+Ai0ItpCtCr38a6gnMF5EDIjJMc6xuIlIrIt1Dz88QkeWh7b4SkaOcuFgiIkpZVwD4GsA/0DKA+hzBIC7sOAB/ivLa56HHvwZQCeBypdRGFXRAKfV3pdTjEef8M4A/6i2giLQFcD6A6wEMFpExmvdOFJHSiO23iMik0OM2IvJSqKdyjYj8Trt9aNvfish3InJQRF4QkR6hqRBVIjJPRDppth8fqj8PiMgKETlR896nInKviCwI7fuhiHTV/DwB4ECoDp8Q2ucnoXLtF5G5IjJAc7xTRWStiFSIyBMARO/PjCgSgzwiZywB8CmAW6K8txjASACdAfwLwJsiUqCU+gDA/QBeV0q1U0qN0O6klKoH8B8Al2hevhDAZ0qpMhEZBeBFANcA6ALgbwBmaYfnEBFR1rkCwD9D/04XkR6h1z8DcKSIdBYRD4AxAF4H0FHz2jFoDl4mAfivUiqg45xPAjgsHIjpcB6AagBvApgbKrNedwIoBjAIwR7Fy2Ic/1QAhwE4E8D7CDaqdkXwu/GvAEBE+gCYjWCA2hnBOvxtEemmOdaPAVwFoDuAPDTX8+HguGOoDl8oImeHznMugG4AvgDw79C5ugJ4G8BtoXJsBGC095OoCYM8IufcAeCXEZUDlFKvKqX2KaV8SqkHAeQDOFznMf+FlkHej0OvAcDPAfxNKfWNUsofmsdXD2C8qasgIqK0JCLHAhgA4A2l1FIEA4kfA4BSaiuArQj21o0AsEEpVQtggea1AgDfhA7XFcAuzbHPCvV2VYnIhxGnrgNwH/T35l2JYAOnH6F6TkRyde57IYD7lVL7lVKlAB6Lss3jSqndSqntCAZa3yilvg01nv4XwNGh7S4DMEcpNUcpFVBKfYRgo+1UzbH+rpRaH/pZvYFgo20s1wB4QCm1RinlQ7Ahd2SoN28qgO+VUm8ppRoBPALNz5fIKAZ5RA5RSq0C8B6AGdrXReTm0NCNChE5AKADgpWnHp8AaCMi40KVxEgEKyggWJHfHKp0D4SO3Q9Ab9MXQ0RE6ehKAB8qpfaGnv8L0YdsHo9g8AMAX2pe+yYUCAHAPgC9wjsqpWYppToiOIwzL8q5nwPQQ0TOjFdAEekH4CQEexoB4F0Eg8tpOq4PCNZx2zTPt0XZZrfmcW2U5+1CjwcAuCCiHj0WmutGy0CsRrNvNAMAPKo5VjmCQzL7RJZbKaVilJ1IF2b8IXLWnQCWAXgQAELz734P4BQAq5VSARHZj+Zx+CrewULbv4Fgb95uAO8ppapCb28DcJ9S6j7rL4OIiNKJiLRBsJfLKyLhwCQfweGYI5RSKxAM8q4B8AOAv4e2+QLBQPAHNA/VBICPAZwtInfrGbKplGoUkbsB3AtgdZxNL0ewE+J/Ik1T0goQHLL5DoCDCCZ7CV+XF8Ghj2E7AfQF8H3oeb9EZYtjG4BXlFI/T2LfaPV3uF7+Z+QbIjIYmrJK8OLNlJ2yHHvyiByklCpBcI7Dr0IvFQHwAdgDIEdE7gDQXrPLbgDFobkQsfwLwEUALkXzUE0g2Gp6baiXT0SkUESmiUiRRZdDRETp42wAfgBDERz1MRLAEQgGceE5b58jOFTxBASHaQLASgADEexd0wZ5DwHoBOAVETkkVM8UIf5wxVcQDCwnx9nmCgB3a8o4EsE5dNMkmNFzPYCCUH2Wi+AcNu1c8zcA/EFEOoXm1N0Q51yJvArgTBE5XUS8IlIQSvzSV8e+ewAEEJwbGPZMqGxHAoCIdBCRC0LvzUZwTuS5Elx24VcAepooO2U5BnlEzrsHQHjNvLkITvhej2AraR1aDs94M/T/fSKyLNrBlFLfINiy2Tt0rPDrSxCcl/cEgP0ASgD8P6sugoiI0sqVCM4f26qU2hX+h2AdcamI5Cil1gMoA7BTKXUACI4YAbAIwQbIr8IHCw35HI9gvfUlgCoAyxFsvLwuWgFCc+zuRDCJSSsiMh7BpClPasuolJqFYB12iVKqAsAvADwPYDuC9Z822+Y9oeebAcwD8BaC89ENU0ptAzAdwWQpexCsn38LHd+flVI1CM5DXBAanjleKfVfBDOWviYilQBWAZgS2n4vgAsAzERwKOxgNAfaRIZJcMgvEREREVFmEZHrAFyslDoh4cZEGYQ9eURERESUEUSkl4hMFBGPiBwO4GY0JyQjyhpMvEJEREREmSIPwXVhBwI4AOA1AE+5WSAiN3C4JhERERERUQbhcE0iIiIiIqIMkpbDNbt27aqKi4vdLgYREdls6dKle5VS3RJvSQDrRyKibBKvjkzLIK+4uBhLlixxuxhERGQzEfnB7TKYJSKTATwKwAvgeaXUzIj3hyC48PQoALcqpf6qeW8Lgqnp/QB8Sqkx8c7F+pGIKHvEqyPTMsgjIiJKByLiBfAkgFMRXLtrsYjMUkp9r9msHMGFj8+OcZiTQmtoERER6cI5eURERPYZC6BEKbVJKdWAYKa/6doNlFJlSqnFABrdKCAREWUeBnlERET26QNgm+Z5aeg1vRSAD0VkqYhcHW0DEblaRJaIyJI9e/aYKCoREWUKS4I8EZksIutEpEREZkR5X0TksdD734nIKM17HUXkLRFZKyJrRGSCFWUiIiJKARLlNSNrF01USo0CMAXA9SJyfKuDKfWsUmqMUmpMt27MUUNERBYEeZr5BlMADAVwiYgMjdhsCoDBoX9XA3ha896jAD5QSg0BMALAGrNlIiIiShGlAPppnvcFsEPvzkqpHaH/lwH4L4LDP4mIiOKyoicv4XyD0POXVdDXADqKSC8RaQ/geAAvAIBSqkEpdcCCMhEREaWCxQAGi8hAEckDcDGAWXp2FJFCESkKPwZwGoBVtpWUiIgyhhXZNaPNNxinY5s+AHwA9gD4u4iMALAUwI1KqYORJwnNRbgaAPr3729BsYmIiOyllPKJyA0A5iK4hMKLSqnVInJt6P1nRKQngCUA2gMIiMhNCI6M6QrgvyICBOvrfymlPnDhMoiIKM1YEeTpmW8Qa5scBNcF+qVS6hsReRTADAC3t9pYqWcBPAsAY8aMMTKfgYiIyDVKqTkA5kS89ozm8S4Eh3FGqkRwGgMREZEhVgzX1DPfINY2pQBKlVLfhF5/C8GgjyjjvfPtdjz+8Qa3i0FEROS6K19chG3lNW4XgyhjWBHk6ZlvMAvAFaEsm+MBVCildoZaL7eJyOGh7U4B8D2IssBDH63Hgx+tb3r+1KclWFDC9Y6JiCj7fLZ+DxZtLne7GEQZw/RwTT3zDRAcpjIVQAmAGgBXaQ7xSwD/DAWImyLeI8pYEjGI+c8frMO4gZ0x8dCu7hSIiIjIBWt3VQIAXl+yDftrGvCz4wa5XCKi9GfFnDw98w0UgOtj7LscwBgrykGUTsIx3qwVO3DWiN4AjC2eRURElAlWbDsAAFi0uRyLNpczyCOygCWLoRORcaGMebh71mqXS0JEROSeHA+/jhJZjX9VRC4J9+QFlKb/jl15RESUZXJzWn4d3XGg1qWSEGUOBnlELtPGdYu2lOOUBz/Fu8u3QylGfERElPlyPS0nqV/76lKXSkKUORjkEbklVKcFAi2DuY17DuLG15bjs/V7XCgUERGRs0r3t+y5q28MuFQSoszBII/IJeF2y1j9ddX1PqeKQkRE5Jr75qxp8Twy+zQRGccgj8gl4cQrsUZlcrQmERFlI0+KRHnvr9yJyrpGt4tBlBQGeUQuizX3LsAoj4iIslCqJNu87p/L8PqibW4XgygpKfJnRJR9wu2UtY1+1Pv8rpaFiIgoVQhSoycPaD2UlChdMMgjckl4NEpAAZc/v8jdwhAREaWIVBjJsmRLudtFIDKFQR6RC/6+YDPW765uer4oSmWSAnUcERGR4wIpUP/NXrnT7SIQmcIgj8gFz3y2MeE2iiujExFRiggEFOoanZlaELm0kBtSacgoUTIY5BG5QE/mMPbkERGR2+oa/VBK4aWFWzDk9g8cOWcqDNdMkQSfREnLcbsARNlIT92RAnUcERFlsRXbDmD6kwsAAFdNLHbsvKlQ/THGo3THnjwiF4ienjwHykFERBTLsq37mx57HezaKimrTryRzdiTR+mOQR5Rioq1fh4REZETfP7mesjjya6oR09jLFEqY5BH5AI9dcf3OyvtLwgREVEMjYFA02M9c8kzSXZdLWUiBnlELtBTV/59wRaUVdbZXxgiIqIotD15erJCZxRGeZTmGOQROWzFtgPYVl6ra9vGFEgjTURE2cmXxXUQl1CgdMcgj8hhz36+Sfe2H67eZWNJiIiIYnNzvboHP1zn2rnf+XZ7i7Vqz3/6K9fKQpQsBnlETjPQOPjR97vtKwcREVGKevyTEtcSkN30+nJs39884mbJD/vjbE2UmhjkETnMyOR1b5ZlMyMiovRxy5sr8MrXP9h2/O9KK2w7diKsfyndMcgjcpiReiPbspkREVHqeGJ+Sdz331pain+aDPKq6hpjvtfoD8R8z25OrgtIZAcGeUQOM1Jt5LAlkSjtichkEVknIiUiMiPK+0NEZKGI1IvILUb2JXLaa4u2AgCueWUJACBgckjlt1sPxHzPzbQv2bYuIGUeBnlEDpq/rgzvLN+he/tOhXk2loaI7CYiXgBPApgCYCiAS0RkaMRm5QB+BeCvSexL5Kh5a4JzxeeuDv7fbzI5yxUvLor5nktT8gCwJ4/SH4M8Ih0a/QFLho0YHdZy0uHdTZ+TiFw1FkCJUmqTUqoBwGsApms3UEqVKaUWA4gct5ZwXyKnRcZ0dibgdCPxSvic7MmjdGdJkKdjKIqIyGOh978TkVER73tF5FsRec+K8hBZ7cfPfY1znlpgwZGMVRrK1cEqRGSBPgC2aZ6Xhl6zbF8RuVpElojIkj179iRdUKJkmO3Ji8eNFRzC5/x3aFgqUboyHeTpHE4yBcDg0L+rATwd8f6NANaYLQuRXVaUVmDV9krTxzHaMOjmUBUiskS0v3q9f9m69lVKPauUGqOUGtOtWzdDhSMyaseBWuyqqGt6bmeQ50ZDp53XQ+QkK3ry9AwnmQ7gZRX0NYCOItILAESkL4BpAJ63oCyU4QIBhV0VdY4P4QgHZ/U+v6njZOoQ/7eWlrq6aC5RCisF0E/zvC8AvRNzzexLZIu1u6ow9bEvmp7bWh+7UK0wyKNMYUWQp2c4SbxtHgHwOwBxJzxxOAoBwOmPfI7xD3yM977b6eh56xqDv563vPmdqeMYXRIhlaqaz9fvwbA757Z63ecP4JY3V6CiNnYabKIsthjAYBEZKCJ5AC4GMMuBfYlso73f2xkU3fj6ctuOHYs/RtC6s6I26utEqcqKIE/PcJKo24jIGQDKlFJLE52Ew1EIADaUVQMA9tc02H6uv322Ecu3HWjx2ua91aaOaTjIS6Hxmut2VaG63ofiGbNbvF5d7wPASepE0SilfABuADAXwWkJbyilVovItSJyLQCISE8RKQXwGwC3iUipiLSPta87V0LUTLtsQqygyAp7quptO3YssZaE2Fh2EI3+QEY1aNY0+LB570G3i0E2ybHgGHqGk8Ta5nwAZ4nIVAAFANqLyKtKqcssKBdlMCdinwfeXwsAaJvnte68aRoH+fwB3Dcn+rTZpkbc1IlHiVKKUmoOgDkRrz2jebwLwXpR175EbtPWhSnUFmmJWNfjVwp/en8tnv9yM1bdfTra5VvxFdpd989Zg1e/DiaYWXLbJHRtl+9yichKVvTk6RlOMgvAFaEsm+MBVCildiql/qCU6quUKg7t9wkDPEo1NQ3N8/DMjkox2pP369eXo67R3DxAPXZXxp/nWFnni/me2YVwiYiIUkasIC8QQOn+4JDNT9aWOVgg+1Rr6vbKDOqhpCDTQZ6eoSgItkJuAlAC4DkAvzB7Xspubg1jNHteox15ARUMwOw27v6PMXtl7HmO3jhDMcM/Ei73QESUfVJpWoEVYtVl5Qcbm5KnZco1S6ZmgyMAFq2Tp5Sao5Q6TCl1iFLqvtBrz4SHo4Syal4fen+4UmpJlGN8qpQ6w4ryUOZz6/YaeV/fVl6DEXd/CAD4etM+jL//47j7JzNtTRwa47n/YHLzHMOV3YrSCiuLQ0REaSAzwp3EbnlzBd5ftcvtYlhK++0iWz7HbGJJkEfkNKWAZVv3Y0VEYhTbzxtxG9y4p7ppEvb8tWXYlaDXLZlWM6ca2uIORY3zXni/K19cZGl5iIgo9WVIp1YTPdeTadcMZOY1Zbv0nzVKWUkBOPepr5Dn9WD9fVOcO2/ETTDH09xOEs4yGU8qj4yIN7cu2feIiCizZVoNkGnXo1/2XnmmYk8ekQF+paCUwpIt5QBazlXTE+wkM/TSqcAw3lpHLy7Y3OL53urmtNasFoiIspeZ+Wml+2sSbpOKi5NnzBz0FG54JvMY5FFaaqpUHL5BCYCyqnqc/8xCAECOt7kAeiqiZAK2QCDxNiVl1Zjx9nf4dut+4ycIiVZP76yoxZa9B/H4JyUtXh/zx3ma8mVIZUdERE0q6/RlWzRTA7z3XeyEX2HlSc4XT5aeoDVTBrDsOMAF3jMZh2sSGSAQ5HqDbSMVNY34y9x1Te/5dQRjydCz0Ozri7fitcXb8Nribdgyc5ol56mqa8SEBz5JuF+mVHZERNTssue/sf0cA7sWJtymtsH+ZYS09FRpmVLvfb2pvOlxplwTNWNPHqUllzryAGk+5yfrdmPR5uYbpF9Hl1sy5dUzDNTo+nt6znP7O6t07Vde42wrKxER2W9ftb57u5ngQE8Al4pDI1OvROZl4jVlOwZ5lJYS3fS/Kz2A+2Z/b9O5g7Qx3Teb9sFv0x0y0XBIf0Dh260HTJ8nsqKurtfXenr2kwuaHv/urRWmy0FERO5rm+fVtZ2ZIEzPNIc9VfUonjEbJWXVSZ/HCH3ZNRkSUepjkJehyqrq8Kf319p+nlXbK7DdhTHd8e6v327dj7OeWIDnvtgce6MEXlm4JeaJwzd3bc/XRc9+3dSTd/kLsYe4JNPhlmi45kff78aiLeVxt9HDikrrjSWl+HjNbtPHISIid+mtr8xUHXpGqoTnwDsW5OkIWhniUTpgkJehxt73MZ7+bKPt5znj8S/xs5darW3vmHpfABU1LSeH7zgQf606PWbGCJADaL65R9ZN4Z69LzbsNX1+rUQtnT49mVl0+OuH61E8YzbKKuuwaU815iUZrG0rr8H+gw2oa3R2HgURUbr6Yd9B1PtS656pNxu0mSBPKf3TGDbucSbI0yUDozx2TmYeBnkZqNGuDCAxuJFdUXvG73dWtnjPa8Fv9cEY8wQ27TmILzbsAQD87u3vWrynJxNZMksoNNo1DjSGnRV1ugK8NTsrcfGzC1u9LiI4+t6P8H//XWlH8YiIMkptgx8n/OVTHH7bByk1DFBvT56e5GCxBJRCQa6+YaHaRGe20jNcMwOivMjftVXbK7C7sg6rd1S4VCKyGoO8DHQwyqLcgYDCr/79LXZXmu/liuTGzU57b9Kef/3uKlz76jLTx79wTN+Y781ZuSvq619t3Bf3mO+v3InXl2wzXJbquviLrNvxnaCssj7hNl9u2NsiM1dY+IsBUzMTESWmHY3R4HAjbTyiM8pr8CVf5oBKvYAp07JrrttVheIZs/HIvPUtXr/mlaUtnt/85gpc/89lmPbYl04Wj2zEIC8D1WqGyf369eUAgHlrdmPWih261qQxyo2b3ZuaYEkbaJz28OeWHN/rif2nkZeT3J/N28tKk9qvOkrQrmX1j3/drio8/2Xi+Yyx6v873l1tcYmIiDKXdjDMht2pMyTR40D66oBSaTn0cZmJNWmdFu6Ze2Tehhavf72pdcN0I9e9zSgM8jKQTzO877/fbgcA1IVa2u59z/qMk27cEjbtPdj0+LGPN8TZ0gZJXnBhfnLLUuqZmG4lH2/yRESO8Wl67854PHV6UVbvqEy8kUn7qhsMVal6F2g3Q0+V+8aS5Bpt3RCrTo/2suPLUpGtGORloMhEHbUNfvzq3982PY82j8oMp4OQWMwMGTFi9srEvaE7K1oPVWyb50yQt2q7ufH0bfJ4WyAicoqeZQQy1cPz1qPeQN39pgPBVaoNH03W819swgPvr4n5+xVtlJAFS+5SCuG3uQwUOQk6MviJNo/KFIfuh/ESvFTUNOIvc61ZMuK70gOmWwsnPPBJq9fa6JxcHilRjBc5edpoS3DxjNktnutdWF3vfA0iokxV1+jHht1Vpo7B0RP6RQ4xXLW9IqWS1aSSJ+aX4G+fbcIf/qM/CRpr9czCIC8DRbbabN53MMaW1nCqJ+/J+SUx3zv14c+wS0eyED3OemIBZtswdzHZHrJfanphnXDja8t1bcfKgIiy3YMfrsOpJueCZ3NPnlEffb8bx/25uRH1jMe/xJ2zrJ0HnikxYzLXwcbbzMIgLwNFVhjxFuc2o6Yh2NW/ZV8Nyg822HIOrQ1xFkItq7ImwLNTrhVrO6SQV7/+we0iEBG56rkvgkmq1u1KvjePPXnGbCtvOR3iFYvrIqs/DaUUnvjE4dwBSJ2pNOSezPrWSQCAZz/f1OJ5VYIU/MlYuHEfht4xt+n5qHs/ajF53A5O3LAihy6acfMbK1o899rQQlbb4Nfd82Y1bfKbaFi/EFG22F+TfEOnX7OEghMZLfVIpyGQqV7Uel8Af/1wfeINrZZMT571pSAXMcjLQOGMmvG8mcR6bVrRUu/a3ZuW6jfySJFLJnhsqL3NfLGw2zeby/HW0vTJQEZEpNfzX2xq0ShoJvGXtidPu0TPywu34PXFW5M+rhlG69t4c+bTjdUBbrh91+nAOZmG8fASXJn0eWYzBnlZ6rdvfWdq/2g3D7uHcqf70AOvDUFeqv9MbnlzhemkBEREqealhVtaPM8xcX/XLnuk6dTDHe+uxu/f1p80w0pGaxa3h5xaGZRYXa1e83Jw0XGnf0TJnC68bMYds1ZZWxhyBYO8LLa7si7pfaPd0MXmjv5EAY0VrWSdC3NNHyMWO4ZrBpxZNcKU1xeb6zUmIko1kfVd5ELTRmjrtgZ/AEu2lGP+2rKm10rKqnDzGysc7V0xWp+63eBY5/O7ev54Pl2/BwBw+7vOBU5Pzi9BTUPyP5NXv3anB5msxSAvwxjJ0vXigs1Jn8eNrvxEp3zPioyYNl5WNvbkERFlosg2u0Vbkl+aKLLR9PxnFrZIJjLpoc/x9rJSrDWR3MUoozVLshlCraoWa00ENE751zfOBU5/mbvO9DGiTcuh9MIgL8M0Gkh+YiY+iHZDt3sBUSfGs1t9Bu08DTuCvMg1EVMRMzITUaax8rYWrdF0Qcne1ud08F5qtGpJtibq1DYvqf0ifz7huWRWSINqNWkvfrkZS3/Q1yBx8bNfN2VRp/TEIC/DGGlNM9MbFy24sHutn3ScB3zYbe83PTYTpMbal+srEaU+EZksIutEpEREZkR5X0TksdD734nIKM17W0RkpYgsF5ElzpacYvFYGHH5AgpFBTktXqs3kcgFAOau3oWbXkt+jVWjjbbJjipJdr9Ln2+5NJSZoYnZ5J73vje0ODq/Y6Q3S4K8ZCswEeknIvNFZI2IrBaRG60oTzbTTuBOxMzfbrT7st3zw5wYmmjnKaz+eQO8AROlOhHxAngSwBQAQwFcIiJDIzabAmBw6N/VAJ6OeP8kpdRIpdQYu8tLrSml8NjHzXPuFpTsTbiEjBF2TH94a2kp3lm+I+n9DffkJXkJZupc7fq8Vi4VZfWopFQbzbJ+d+w1hyOlw7x/is10kGeyAvMBuFkpdQSA8QCuj7IvGdBo4C8ymaDpiw3BCcTRgouaRnu79dN9CIWZ4sfal3PyiFLeWAAlSqlNSqkGAK8BmB6xzXQAL6ugrwF0FJFeTheUogso4KGP1kMphc17D7bqRTJLb2ZKI416ZmcHGA/ynO3JA4ArX1zU9LiqrjHp40RqNNBYrksaV9PpMCWEYrOiJy/pCkwptVMptQwAlFJVANYA6GNBmbKWsZ48Y3+8FTWNuPyFRfh0XVmLSeFhkx/5wtDxjEr3gMbMcM1Y155OP5J91fVR55kQZbg+ALQpZkvRup6Lt40C8KGILBWRq6OdQESuFpElIrJkz549FhWbwsLBlVLA7982t/xQvOMn0mBgzr3Z4aTGh2smeR4TddjK7RVNj6vrrWtkfmSevoXL//lN6+9B0ThdTVfUJA54tWs8xrN9f63Z4pCLrAjyzFZgAAARKQZwNICoTWSsxPTx2diTF27RWbb1gKH90kmqxkyxPqp0CvKe+nSj5S3gRGkg2rftyL/ceNtMVEqNQnBEzPUicnyrDZV6Vik1Rik1plu3buZKS01qG/wonjG7qa4MKIWDFgYTYQ3+gK57ud4hiat3VJgP8gzWLRc881VS5wlYVOtaGYzsrNC3vNTzXyTOUF4fsbSDEwnkXtUZfOphJgs7uc+KIM9sBQYRaQfgbQA3KaUqo52ElZg+Ru4fRlvewul0G0xOCE+WlWPunRRuVTNzb48VkKdD72a4iJGJBYiyRCmAfprnfQFETpaKuY1SKvz/MgD/RXD0DDkgXNeFh1PaNQVab1bs/Zo5aLEopTDtsS8NZdqOehyD22/ck9wcRauqsA1l+ueZJaI3ENusY15m5OgqJ6psK7+jiQR7/ZZvO2DZMck5VgR5piowEclFMMD7p1LqPxaUJ6sZ+dK/2eBN+aGPgkMY6uKkKt5+wL6u/f01iSs4s+xopR1xz4cA7AnIUj/Ea/5iFLl4MFGWWAxgsIgMFJE8ABcDmBWxzSwAV4SSlI0HUKGU2ikihSJSBAAiUgjgNADOrahMAJqHU5bur7Hl+I3+gK7A4oCOOjB8GLPDF53ocQKsqxf/s6zUkuNYLXJOmxMNs0ZGdCXiDfUIb95rXRBNzrEiyDNTgQmAFwCsUUo9ZEFZsp6RlkajE2rb5nkBAMP6dIi5zcSZn2D9bnsWbHXm5mjfOcwcOZ178sJltHsdRaJUpJTyAbgBwFwE552/oZRaLSLXisi1oc3mANgEoATAcwB+EXq9B4AvRWQFgEUAZiulPnD0ArJYuI4M94yc/OBnWL0j6mAjAMk3Ejb6lK66W89olvD99quN5hayTuZurXeOWovzWFQtBJR1C6JbWVP5/ZFBnoUHj8HKRdffXBoMntPgqwZFYXr8lFLKJyLhCswL4MVwBRZ6/xkEK7CpCFZgNQCuCu0+EcDlAFaKyPLQa/+nlJpjtlzZykjrm9HhHOF+mFveXBF3u9Me/hxbZk4zdGw97E7la3fLpZnDx56Tl/p33qYgL/WLSmSLUJ02J+K1ZzSPFYDro+y3CcAI2wtIUYV7RCLnVcWyv6YBhfnGv1Y1+AO6GsH0rJ1nVRCRzP361v+uwqXjBth+nlgafAG0CTVGpwo3evL260i8YtSsFTtw7qi+lh+X7GXJJBkTFdiXiD5fj5KkzTaViOFx2y4v9pLMUNCyyjp0b1+ga1uzi88mYqYnK52za7688Ad0LsxjPx4RuWrNzkos+WE/Lh+vLxAJNyzqrRuSTb3v05l4paYhcU+eZSMmHLphW1XeghwP6v1+ALmWHM8qkZlT06HOjubTdUx4mI4sWQydUseNry3Xva1VQxtS2dj7P9a9bby5hlYw1ZMX4/XPN0RfkmD1Dv3BvhNmf7czfWs3IsoIj328Abe/o39KY7gnb4POxaP/7z8rkyqXL6B09fC8uGBLwm2sus06Nbzeqp5HEbEs4YiRn2GieZKRU0Difc6/eX05fvP6cv0nJ0qAQV4Wa/AHsGhzOXw6hm1W1/uwwkB2pf9+a+0kaD1ljKWsUl86ZCNrECXDzNBKFaNoj328Ierr0x77MulzERFlIo/BFcLDvTDXvrpU1/YLNyU3D67Rry/IAxIv+m3VcEArs1XGY9WUg9pGf9IZPs3YU1Uf9/2AgSDvP99ux3++3W6qPOc+tcDU/pRZGORlsdL9tbjwbwsxe+XOhNtWG1y+4Nevx5+3Z1SNiV42vb15dnc0mevJS/9esPS/AiJKV/6Awk4DQ/5/9tJi3Pve9zaWqJnPH9A95/zFL7fEfd+qnrELnllozYESsLLevfLFRdgSWtbgL3PXGmqYTlaiIbovfbWlxfN4n4/BNoioMnkdYzKOQR7pav1yO4tjfaP9a/PZeY0lZVUms2u2fi0dkq6EiXC0JhG5580l25q+ABfPmI03l2yLu/28NWWYt6bMgZIBjQH9zXj+BNFgZD1mZhSM3QIGrluvRZvL8VXJXjw5fyOe/nSjxUdvLVFSnue/bLmYeLx62+wC9nZLp+8cFMQgj/C/FZHLGrZm15/2ul1V+NlLixNuFzl52Q52nmLSQ59jX3X8YR3xRAtAnfiZWGVnRR02hdbZMbtILxGRXnur6zH10S+wLWKNu282l9tyvmSCqgad2TuBxEsfRQ7t17PsgpWMBAIN/oAlvVdav3v7O/z4+W8AAB+s3pXUMYzUrOc89RX2Gqjb41XbKR7joXS/fesgkz0Y5BE27z2IXRXx561Fjis3a9X2Cry9tBS3v7NKV2upE4GB1dcY6aWFxtcQCosW5MVb089rdc1pUlWdD3NWBivcH/Y5P2+CiLLTxrJqfL+zEk/Ob9mr89bSUiwo2Rs14dbSH/Ynfb7FW4zvayQrZ6KqMLJvLDIoTHYtP72MZMFu8AdSrq5Kxpg/zkOJzjmM8YJgMRnlrTKQXZ2yA4M8AgDsOxi/JWqPiV6oaM54/Evc/OYKLNqirzXVmZ681O0ZizZCJ16iGG9KNwmmctmIKJPECyIuff4bvBMl0cV5T3+V9Pnycozf34w0YiYa4BhZVUY2Xh5551ys21Wl+3xGHfun+di6rybxhggu45SSQxST+C6gdy1FO+fkRftdtlIqflQUH4M8AgAcrI99g1JK4dynkq/09EiUjdNn90rosHe4plnRrt8fp/VX+8Xmw9W7cE4KZdxiRUFETknUU7R5b8uRBWbnHeV6E3+t+njNbtz02rdNz306e/JyPIKhvdrH3SaysbIxSsVWul9fEJasqnp9i3E3+lMzyEvmN0Bvj6Sdc/LqDAz7TcYcHUn6KLUwyCMAwIV/i51Jq7LW/jH9q7dXxn0/3tBEq6RqT57XI1F78uL9TLT1zQerduHbFMq4lXpVOhFlqhxP/K85f/t8U4vnZusaPfu/vawU7yxvnguv95w5XsH63fF74SLrsXnf7261zU9fWoKHPlyn65zJSNQm++WGvaht8KPRp2xv9LN7GkaYRwTf74j/PQYAXl74Q8xAL9GPQikVcz3fRn8Ar369NeH5zbh/zlpbj0/WY5CXQey6mTU60IuWqBVMb0unGU5VBkbl53iwp7r1nMl4Q1i1b9ndukdElKr09LAs0UwbqDe5oLaeUS+RPTZ6h2vWNQZazS2MFBk/aJcR0AaIy0vtm7+VKDnMZS98g38v2ooGf8D2IG/NrsSBlxW+K63A1Me+SLjdE/NLYs7BjNaTV1nXiO9KD6CyrhFH3fUhhtz+QdN789eW4eJnF2Lz3oOOfEei9MMgL4NYEYw9Ob8EX0cs6OrEfLhEE47tXqgcSN3hmjUNfpz3dOue1ngVqV+pptbCOgeWnzAiVXtMiSjz6Akizn9mIcoqgw1pDSaDPD3CX+bf+XY76hr9ltaxkbfXwT2Kmh5rk6LYme8kXobR8FDRgFJo9AcgNo/tSGYIZJtcr+F9ahr0j3haHmv9voiiHqz34a5Zq3HWEwtw2sOfo0qTNGdnRS2u+sdifL2pHM9+vjFhYE3ZiUFeBjGSoSuWv8xdh0fmrQcAlB9sQE2Dz/ZsXACQaBqD0cXYk5FuwUe8OXkNvgBeXxxcBypeK7EbvZdcQYGInFJ+sEHXdpV1PiilHArygv+/6fXl+Oj73ZYGeZH1mPZ6tIk57EzOFa9R9gXNunGNDvTk/esb40MYTz+yJ3IMRsF3vLu66XG9zw+lVMxgN9b0GG1Auv9gA468cy7+syz4mWkzoK/aXoEdmoA9P8eL/Tp/z81K1RFPFB2DvDRz1d8X4UBN9D9mqxY9ra73YVt5DUbd+xF+//ZK3D9nTVLHeXd57ExPBbktf/USVTjVJgNNPZVougV5iZLRrNpRAZ8/gL3VsW/+brT+GVlTiIgoWe8u345LQ2umJTLpoc/w5PwSjH/gY5tLBXg0AcTCTfssXbA8sh4Lz+H6fkcl3tXMA+xWlG/ZOSPFmjcGAAWhXrJwT57dXvk6uaWLzCRBOfy2D/Digi2Yv25P0seIl/H8t2+uaDG65x9fbcFxf56f9LmMcGJUFVmHQV6amb9uD9bsjD7x+kCNvoxWsSwo2Qsg2GI09dHg2PLqukbsrkzuS/mNry2P+V6P9gUtnucnGB5htjLQs386xHg1Db6mYZiJgtIGXwD/+GoL1uyMPSfBjQXV9X7pIiIyY7WORBhaqxIkALOKtpfoX99sRUWdubpbK7JaqA0FXJHzxQ7t3s6yc0aqbYhd3+aGrl0poMGXmpWu0vw3WSVlVcjPMfYVO/xrccKf58cdmbXGxiUwEtlhYB1Ech+DvDQUa52ci56NnSFTj/BY+b3VDU1jv9vm5WClxQtsrtpegR8i1tHpGRH0+fwB7KxovpmYDUb0tD65EfAYNfSO5uEbiTKy1TT48WmclsT3V+7EZS4FXGbTlBMRWe2D1bscOc+RvTu0eF5Wpb8htW1e/AZRf0C1mFP2j6+2RN3OzozVuhpV4czSSABQPGM2LvrbQlTobAgPBBKtRpjYp+v2oF1BTlL7/lBeY7rR3i4z3l7pdhHIAAZ5aSjW9+Nke9zC9ke5qRhtidJDO7Y87OY3V7R4/uKCzZjwwCdNz81WSI0R8yy2lddgZWlFiwnQ5TGGwaaK8JCe8ML0iYLS3ZV1+DLUOxvNq9/8gCU/7HdlSYNdla1/B4iIMt2+6vpW9+4tEWv1xZNoBEdAqVZJVaI1qj396UZsK6/Bd6UHWpTNCg9+lHh5hh0HarFi2wHUNjiT/fmbzeV4d4W+xcIDSiU9sif8s95ZUWd4fWHtdzC9c0mdVu/zWzq8mOzFIC8N2dUJ8nKUFr/8XOt/RWJVUtpx/J+vDwYnS3/YjzeXbMOLmsnayYgc+vDTlxbjzCe+xNlPNi8SHmuuY6o49Nb3ATT3eiZKmZxoHmM466Yba9Fu3WfvYrxERG6NGIiVvGX1jgqM/uM83PPe9y1eN9KGmajzK6BUqyyNOyMaVgVARW0jnv5sI856YgHqfX4sKNmL0X+cp78gcWwrjz2kL3ypLy/8AX/9cL0ja+CGfb5e3xw5bXZqo466+0Nd2zX6Azj8tvexO0aD53NfbIr6upu8IlhRWoFDb33fsgYBsheDvDQUHkhQaeE4fgDYEaWHTc8aQ4lc9vw3LYZvxLqn/3vRVjz44Trsq65v6oE67+mvcM9732OtiTHoXo+0+llFq1jSZZ2ZcAufdhJ9NImWTjhrRG8AZmceJOfaV5di2J1zXTgzEWULt5aPqY2ReOS/y/T1JMXT4A/gqLti3zv9AbRaluCYmZ+0eB6+54czT64srcCe0JDRB95fY0kGxa82Rh9F4ua0iDHFnVHv88cMrMLMDNes0pkJ/K5Zq1HvC+CJT0qivh9zmQUXaRO1jf7jvJRvGCcGeWlJqWAr0FF3fWj7UId4E6j12LL3IL4s2Ys9VfVNaYNjxY13/+97PP5JSYthmoD+m2YsbfO8rW7q0RrpnGxRNOOe977Hki3leHFB/N7NhohF0KvqGlvMc2wTmtvhRmP3/ppG0xlTiYjiOWhg7TIrPTpvQ9TXnzc5IiWsss6HS579OuqwuWSyRJ//zELc9PpyAMDfPtuE7yyYh//t1gOtXlu7qxLPf2HNzyAZM99fi/tmr8G4+z/Gos3lMbcLKPvXzf1nKMDWZv9sn+QcPreMvOcjt4tACTDIS0O/fWsF/vzBWgD2p7N9e1mpqf1P/OunAIItieGbaqLeQTuuKXL4zOYocyB8/gByvW7MUDOuojZxL+72Ay0D25H3fIQJD3yCb7fuh1IqaiXstEBA4YlPNuC+2d8jEFBRPxciomR0aJPrynlfW7wVK7YdsLUha+Gmfbj6laU4+p4P8b0mi6gVPWXaaQzJ+svcda2GPE5+5AvXU/C/vDAYVP1nWSnqff6oQ2ud7m38cPUulFXW4ej+nRw9L2W+9Go2IADBBCvPhVrDnJgA6xFrW7WcvoGGez4T8QVUaJhL6vfo7Yuz9l0s4Z/7OU99ha//cAr+vcj4IrFWawwE8NcP1wcf+xX+8dUW5Hk9mHPjcbam+CaizDeoayHycjyOLHCuVdPgx/RQoPSvn4/DMYd0RZkNyaY+WVsGAFi5/QCG9m4PIFjfpUpTZWWtDx3auhNoJ7Lkh/04/LYPMG5gZ7x+zYQW7zn9HeXqV5biR8Wd0mIZJ0ov7MlLc04MMbTyFD5/wPEbaL3Pj2tfXRZzcfYbX/sWjf4AfH4Vc3mKVBOe85Fs0hQnFvzVo3R/8/DRcKrvBn8Akx76zKUSEVGmaPQr19vsfv7SEgDA2Pvtu+dql2Dwp0CUFz79iHs+xD+/SW4xcruVlFUDCGbdjOTGvMHFW/ZjyQ/7HT+vWXpGFZF7GOSludnf7USjP5A2C1Q2+AMtJu86IZxZM9bi7O8u34EV2w6gpsGXFmvlAcCds1YHH6RHcWM65UEGc0RkD18gkNQcNSsdbPDbPndeuwSD29cLtKyWbv3vKny2fg++Kz2Ajinaq6d1sN6Hpz/b6HYx0sYIndlEyR0crpnmZq3YgaU/7MfslTvdLooua3ZWwh9QKMzz4qBD6+NoXfS3hXjlp+Navf77t79DXo7H9snWlkuP0aVJmfH2dygqyMHPjxuEA7WNOKxHkdtFIqKQRZvLMWZAJ3gsyMBsF19ApUTQM+XRz209/tvLtqN9m1z8YcoRlmTGtNqVLy5yuwhxlVXVoXtRcGmiZKZCZLviGbPRq0MBPr75BLTNY1iRStiTl+aWbzuQNgEeAJz39MLQsEh3fLO5HIfd9n6r1zfuOYg1O5NfpsEtZr+/pPD3M7y2eBue+2Izxt7/MU57+HMEAgq7K+uaUn0TUWL1Pj+UUqhr9DclNjpY78OJf5mPep8fy7bux4GaBuytrkeDL4BV2ysw+ZHP8ZvXl6OkrApX/X0R7pq1Gk/OL8H8dWVY+sN+NPoDuPBvC/Higs2Y/sSXuOrvi1A8Yzb+8PZ3uO7Vpahr9OPW/65E8YzZWLHtAIpnzMYLX27GDf9ahr3V9U1JuKrrfaiqa2xK0LF+dxU27z2IVdsrUFZVhxXbDmDxlnJU1DZiyZZyHB66dwcCCtX1vqYesle//gFnPP4Fvtiwp0Vad58/+UWtrbTFgXVB/75gC3760mKc/8xC28+Vacbe9zGe+GQD1u+uwppdlYl3oFZ2VtRh6B1zcfkL3+DYP32CqrpGbCuvwQtfbm4aafbFhmCPblijP4DaBj92VdThHws2Y//BBtT7/Hh/5U6c/vBnWL0jmOV1begzeeGLTbj7f6tRXe/Dr/79bVNiowZfoEWSn8g5uEopNPoDmL+2rMWoN6UU6n1+1DX6sa+6HmWVdVgSut/sqqjD6h0VeGDOmqjX+8CcNfjr3HUAgCNu/wB7qupRWdeITXuqUdfox/rdVVBKBdej/HQjlFJYt6sKj85bj1/9+9um42wrr8Hwu+biqU9LmrLAr91ViT1V9Vi2dT8embceZVV1KD/YEHVobOn++PcWsWKxUBGZDOBRAF4AzyulZka8L6H3pwKoAfD/lFLL9OwbzZgxY9SSJUtMlzsdFc+Y7XYRTPvzeUfhrv+tRo0LPXmUGTbdPxW1jX4U5rPVMNOJyFKl1Bi3y2GGk3Vk1+IjVLuL/xrzfa9H0mZYuhld2+VjLxdsJspYeV4PzhvdNyWSyLlp50s3oX7nhqhN9qa/IYmIF8CTAE4FUApgsYjMUkp9r9lsCoDBoX/jADwNYJzOfUkjXUfnaTN0/jGULp+SZ9XvQSr/PsXL6jro/+YAANrkejFpaA/8b8UOfPTr4zG4RxHKqurQoU0u8nO8qG3wI8craPAFGBCSK5yuI6vqfYiXlzYbAjwADPCIMlyDP5D1AV4iVnzrGQugRCm1CQBE5DUA0wFoK6HpAF5WwW7Dr0Wko4j0AlCsY9+0UFZZBwWge1E+AirxWnBAcMiJCCAiLbqaw8+r6n1om+uF1yOoawxg38H6lP1Cnoj2e0Vlnc/t5F9pL5nfg2gBXSr/PikkDkJrG/3434odAIBTH2457+WqicX4+4ItTc+vnDAAh/YowpG92+OHfQehFDCqfyd0aJOLitpGdCvKh0JwCEdAAfk5HjT4A2jwBSAA8nO9KKusQ22jH93a5Tc9L8zPgdcjyM/xwOMR7KqoQ99ObVCQ48XBBh8KQ3MUPKEeFE/obz78GAg+b/QH4BVJ6TlOlBRH68guhXm2XAQREaWgOEMyrQjy+gDYpnleimBLZKJt+ujcFwAgIlcDuBoAvO27ZcSwxWyWysFFpkq3n7nZkeTaAA8AXlqYmqm8Kb68noeOdrsMJtleR2rrx07de4NhHhERWRHkRWt2jvx6FmsbPfsGX1TqWQDPAsCIo0erV284Frk5AoHA6wlm0crxeNDoD6ZM9oQWEBNBaJtgn0C9LwClmucleD2CXK8HgdDETK9H4BFBQAUnbCuFUNZF1TTMJdcrqG0IbusLBOARQen+GrTNy0Hnwrym/bVyvR74As2TQXM8nqZ9/aEevUAgmP44L8cDf0BhT3U9ehQVwB/KELbjQC2u++eyaD+etJPKwwQpdZj5PRnUtRCbNKnFuxflY0S/jhg3sDM27K7GgdoGTB/ZB0UFOdhX3YA2eV4UFeSg0R9MUtG1XT7qGv2oafBja3kNiru0xd7qelTV+VBUkIO+ndpid2Udigpy4fMHkJ/rQWFeDtbsrET39gXo2SH4t9sm14tcb/DeBASHy+XleFDX6EdejgfVdT4U5uegut6HXK8H7QtyUO8LIMcrqG8M7hO+p3g90nQ/Uir4uoJCjkfgDwBeT/Cn5gsE16MMBIL3L5Hg8Nd6X+i+5W++9zVTaPAFy+YLBJDjEQCCusbgkFeBoN7nR47HA09ot5zQA18gAEHzPbV5JINqOmcgAHg8wX0CSsHnV6FySdP9VangNvk53qZSHfbg1lVJ/gqkCtvrSG392HfwMN5aiYjIkiCvFEA/zfO+AHbo3CZPx76t5HoFw/t2SKqwdhnWx/7yjOjXMW2DoxyPtFi4PS/Hg/qIDEikn5nfA+2+qfz7FG9OXjR/Om84zhrRBxv3VKNfp7bo0DYXOw7UwusJzsnr17mtfYXVGDeoiyPnyRbK15Duk6scrSP3VNejV9JFJSIyr7mZj2wnEnOOhxVB3mIAg0VkIIDtAC4G8OOIbWYBuCE0n2AcgAql1E4R2aNjX9JI1z8YbYB33znDcN/s6GlpSR8zvwcqxuNUEy/AW3PPZOyurMOALm1b3d+0DS69O7axq3hEejlaR3pi1/dERI5I5e8W2cR0kKeU8onIDQDmIpji+UWl1GoRuTb0/jMA5iCYGroEwfTQV8Xb12yZKLUVaIZikbuM9pa5adntp6KkrBpKKbTJ86K4a6HbRSJKyOk68sje7fHZ3acjL8cDrwh2VNSiqCAXlbWNKN1fiwmHdMHmvQdx0l8/xZLbJmHe97sxdmBnVNf70LFNHhZvKcfNb67AkJ5FuPPMI3HJc18DAIb0LMKJh3dHgy+Ai8f2w2kPf45pw3tFXaf1g5uOwy1vrsCq7ZWYMWUIZr6/Fsce2hVfluzF/244Fl9v2ofLJwzA+t1VoWO3R65XMH9dGXZX1iM/x4MhPdtj3prdEADTR/bBhrIq/PSlJdgycxqq6hqxu7IOjX6FIT2LcNGzX2PR5nLcdeZQnDSkOwZ0Cd4bHvpwHR77pMTsR5h2igpyUFXnc7sYaeWQboX4zamHY+Oeajz00Xq3i5P2PrjpOGzdV4OXF/6Ay8YPwORhPfHc5xtRkOvFaUf2ROfCPGwrr0FBrhfrdlfhiU9KcPsZQzGgc1vc+973+M+32/Gn84bj7KP74PXF23DRj/rh8ue/waa9NXjvl8fitIc/w6e/PQmdC/Ow40AturTLQ36OF9X1Puw/2IA+Hds0dS4cqA2unfnCF5txwmHdcMyhXeHzB7C/phFVdY3I9XpQ0xBcL2/VjgqcdHh3bD9Qi5Kyaryy8Ae8ds14tC/IbXF9pz38Gbq2y8djlxyNMX+chy9+dxK8HsHW8hoc1qMI89eWYdpRvVB+sAG/fWsFXrpqLN5eVoqnP92Ibftr8f09pwMA1u6swvQnF2DcwM64/qRDccwhXTB75U706tAGG/cEz//oxSNRfrABvTq0Qed2eWgXyhSulMKizeUY/6eNMedxWbJOntO4Tl6zru3yUJifgx8cWGzVCo9dcjQCAYVb31mJg/XurJO3+NZJ+NF981q93jbPm3Zr96XycEuz+nRsg50VtXj3+mOxs6IWpx3Z0+0ikQsyYZ08J1lRP9b7gvfB/DgNcv9etBXnHN0HBbnepiVxIjPDhudoWqXe549bpkhPzi/Bgx+uS5uGLDP6dGyDOTcehw27q3DVPxYzyDPg7euOwegBnQAAW/fV4Pi/zHe5ROnpy9+fhL6dnJkWQc3i1ZFcOCrNXTCmH35x4iHYUFaNc5/6yu3iJHTKkO74ZG2ZawHelpnTor7+9nUTMH/tHjw5vyRjg6Z08+XvT2oaiplqc3CJMpmeQOqSsf2bHsda9sPKAA/QVy6tXK9ETYTmtFV3n45hd8617fhnj+yNRy4+GkDc6Tmu+X/HFGNEvw74/Vsr0eBPvbn44QAPADq0yY2zJUWz9LZJ6NIu3+1iUBSexJtQKrtyQjGKCnIxqn+nxBungLwcTyhrn3PCZ7v9jKExtxncowjt2+RY/qXELneeGbqW9ChuTG9eOyHme6n4ZYWI0keOxwO3b+l5OR4U5tk7RUGbVCrV6rDxgzrjrrOOxDlH903JAC9Sh7a5uPaEQW4XI2189OvjGeClMAZ5aS7Ha/8N3co6I8fj/GLPhfk5mHnucFx1THHU9zc/MBXtC3JDXwhSq4KMpU1u8EtDsg3U7/3yWAtLk7zhmiQpAzVz7N64JnbwR0SkR45XXG8suvusIyEitt5ztQmmvCIpM4b/6z+cgteuTs17eVFoXlNxl9bDC8NLwzhpcPd2OKxHO8fPa9bgHkVuF4Hi4HDNNDV2YGcs2lyOXAduRlbOZxCRYCXkIAHQtV1+zOAy/CUg14GA2SqdC5Nf7viRi0biiF7tm36H3JTr9eDwHkUoq6rDBzcdhwUle3HCYd1TrjWaiNLProo6V5bqKcj14J6zhmHswM5NCaLsWGapb6c2KN1fi0GaBrIUivHQo33q9vBMP7oPbjxlcNSRRU7XP+eP7oufTByImR+swfrd1Y6emzIbe/LS0Ms/GYuXfzIWgP09ecce2tXU/v/62TgAwGe/PbHpNccrIAkOmdEaGCUzo9fjSYvhJADQRsfwn94dC9C5bfP8goV/OBn/u+FYnH10H3g9gvNH9bWziLp4PYK5vz4e395xGvJzvDh5SA8GeERkiZ0Vda6c99hDu+HCH/WzPQPwxzefgI33T22xNqcV98+HLxph+hi/OfWwVr2oz1w22vRxzTr+sG4AgNEDOqJbUT46RWkwdXpEz18vGIGhvdtj056Djp6XMh+DvDQkAhTkejH3puNRmG9vZ2y0YMiIY0JBYo/2BU1JT/wxugZH9uuIIT2LMP+WE1u8XlRg7hp9ftWq5yvaPdyJoa9WuHRcf0w8pCsOTzBMItfrQXlNY9PzXh3atEhg4mYygnY2/94SEbW1eS5cLH85/6ior59nYcPalpnTkJ/jbRXUJROg/Pn8o3DbtCMABJfKOGtEH9PlO7R766GHk4f1xC9OPMT0sZN17QmH4PGLj8b/bjgW5xwd+7PweqydphJNp1AD7MlDuje9Vrq/1t6TWuzrP5zidhEoAQZ5aSh8Ez+8p/1joX0WjNXcMnMaCnKbK9tYN89//XwcPrjpeAzsWoheHQoAAH86bzguHz8galCmV22jH13atQzyolWE6TJcc/ygLvB4BH+O8UUirE1u/C84s1bsAGB/ZRbNXy84KmamUyIiKyS6B9p23hjB5bmjzAdPuV7B5gemxnzf6wFUxHiZyPmA4Vv+qUN7AADOPKp309qCH9x0vCW9gVOGRV/yxs2RGjsO1KJD29yE2ZqdyBvwxe9PBgD8YcoQ289lFe1XpPdvPA49Q9/TKHUxyEtDdt1+fnv64a1ea7BhPkOsifBt85p7d84fHWxlu+hH/fG7yUPwp3PjBzSJ5Hpb/qr/8exheODc4XjwguZhKe3yUzt18pLbJgEAdlYEW/sSVZaJWrHDAbwbyWbsmJ9CRKTlVkCRnxP9q9XEQ7tizq+Ow02TBid9bI/ETyYjUSblHdm7fYvn4bevO/EQPHrxSLTJ8+KUId3xvxusSw4Tq4zaAHPa8F6WnU+PaL2L0XhEkq4X1/1xsq7t2uXnYMvMaTETlzx2ydFJnd9OfgX069wGi249BUf0ap94B3Idg7wMYnZpgmjJPMKL4lqpfZThlw+cO7zF8+tOPATvXD+x6bnZyjoyyBs/qAsuGdsf541uHrLRvSh1J4kDweQxQPO1JBpe2rNDQdM+0YRbWq3orTWqZ3u2ABKRvdwakB4vCBvauz26RNS1A6JkeIwlUV3oFWl13dHKM2PKEIzq3wnTRwZ7Fz0esWw9Uj1z+kb174SLftTP9HQMI84frW+4rJnkcNq1HN+Ks0RQNNqfReTvSKroXJiP7kWsv9MFg7x0FOP+8/Z1x5g6bLQgr67Rj6EWt9iMHdi51YKj1XW+Fs/b5uVgZL+OTc/NzpfL8yb+VU+H5RM+++2JuGz8AACJK6LCvBxcHto2mqsmDsTae/W1OlotR8fnQUTkpFhDDK22dldVi+ddDXyhr2mI3/DqEWmxzZUTotcBdma51rMEgYhz8+C3zJyGLTOntVhqIh6R6PP2jbhkbD/DmV21DfWpuij7rVOPcLsIZAC/aaUZr0dQ3CV6MpT+nfW3BkYz6Yjg+PyhvdrjtavHAwAa/CrpJBmxAgwRaTX/raquMeq2YWbXrdEz3y4NYjwM6FLY1JOXqEU3P9eDnx03sNV8RC03AtunLh3l+DmJKPv0NjhnqIdDIwzqGpu//B/avV3cERdGRVaV4fmBT0fcd7cfsC/JR7y5kM3TBPQ1vrrBI2J6WkzXdvmoimi8TiTcAxvMYxD7Z9O3k75g1Q7DOdUirTDFXZrZeH/sCddmW8XCQUO7/ByMH9QF835zPDq0ycPuyjqc8fiXho9379nDYr63t7qhxXN/gkyPhfnmJtDr6TlKt9T9icrbu2MbFObn4Iie7fFlyd6o25gd4puMQd3sTStORAQAV0wohi+g8MfZaxJu+78bjsXhPYtw0Y/6YcqjX9haLm1m419POgyzv9th2bEjG+7CSc+mDO+FQV0LsWlvME3/tvIay84ZKd4SP+HeLYG0mkZhh2QWGPeYXGvwqxkno2u7fMNLMmm/BrUviN2Td/85wektV7y4CABw1cRinDmiN8596ivjhTUocjkqSm38tDKIFTfMycN64pJx/QAAh3YvQreifEtbGWNJdC90Ytx+OgzX1EoU5F1zfDBVdbxeTCeyiEVyI7Akouzj8Qgm6lzrtWPbXOTleByZmx0O8k4/sgfGDuwMr4XDFiOrMe2X8tvPHNr0OFHDqhnxAoEzR/QGECxnrtcDu1fyefRi4wlMvtiwF41+YwX7/eTmLJm9O7ZBXo4n5iioP8ZoAFeaH0b39gVYMONkjB7QCUDLLNjHH9atxcitusYAButMKmNWujWGZzv25GUQK4Y+RFus1Ilx8yrBnV47mdkuqXrvKsj14KELR7Z6Pd4Q1ja5zesnOfGzMyLdgmkiSl96gojnrhiDfqEvzU70VITXiv3b5WMAmJ+OoBV5fy2rrG963NGheV7xGvLCc+1FBHk50mq5B6slsx7snur6xBtFMDKH7sIx/aK+HlnUPh3b4JGLRmLxlnL8qLgzjvvz/Kb3irsW4vqTDsGT8zfi7JG9GXxRVAzyMohdvTJO9LzEWiA9zIkbWLyMaG4SCIb1bj0OPt73Au2lxBs6Q0SUyRLVLUDzenGA+Uaxv13euqE0UmTgobeOzfN6cMZR8ZcdiKzG8jVzu47u36npcS8b1zhLVF//etJhOOOoXqhr9NueAvWQbs70cI0a0DFqI3mkw3u0izm6JlpA2q9z26YGiJL7pmBnRV3Te789fQh+e3qwB9GOTOiU/jhckwDET4ZRFGdsuFUSLarpxELlqdoSVtvojxrQxWv91dYV4eEeqcKttOZElH0SDUs8TRPgAebrGj3B0zGHdMXArs1zk/VmG/Z4gkP14m4TEeVFS4D20IUjcPdZsefMm5WoLr1x0mD0aF+AXK8Hdq/gUxAnCUwsyf4GTNaRnfXln46L2aCc6GeR4/U0BXyR8nO8LRor7DB+UGdbj0/WY5BHAOKv0+P1CB63eWHOn0wcGPd9r4XDWWJJ0RgPQPRKM15Fqm0RvGz8AGyZOc2WciXD7jkYRERh/kD8Cd8nD+ne4rnZER2NOpJtXDZ+AObfcmLTc709eXWNgabEKbFEBnnR5uoP6tbO1mGpsRaDjxQM8lKwQkjid0BPj3GiQ5sdumo2w3oif70g8fqHlFoY5BGA+JmcALRYs84Kj148EheM7otObYPnTTTU1Ikho6k8VyxaQBdvDqYbC5zrl8plI6JM4ouTQOP3k4fglCNa937MmDIkytb6NPiM39+M9B4mGpYXeaTIumPeb07ACIsWPY/m3z8fr3uIZF5OigZ5SdC76EK87xlmfxSJennNypCPKqswyCN4PYI+CRYJtTr+mT6yD/5ywQg8fdloXZkznUj+YneMN/nI5BfajVYxxPuZ6G1VdNJRoS8WehekJSIyK3y/OefoPi1eP29UX1x34iHoFiWb5rUnHJL0+Yb1aW94H73DNYHEi5hH1hWR2x/avZ2t88/HD+qs+/j5OfYP13TCrBsm4oheRbq2jRfkmQ14T7A5yKP0wyCPcMWEAQl70uzq5Ro/qAtW3nV6wu2cmC9n5zle+elYDO1tvPIPixrkpfL40giH9WjXVAG1zWO+JyJyRr/ObbFl5jQMi1jE+ZDu9qzXmcwcdr3DG4HE9ZREHKqtyTVmjTISQOZ6PZY3SN5+xlA8d0Uwa6ndPVtAMNHOUX076r7ueB9fqveU2Zmsh+zBII+iZm6M5PZQxnyvE0so2HeNxw3ulvRkbiB6xZCq2UCJiFLNpeP6o3foS+raeyfj2uPj99YN7FqY1ELaycj1enTXD4mCvMh6LNWW0NHyevQOctRv3MDOOGVIdxzSrRA/Hts/qWMYKVPfTvFHpkQm9olXb6f60FUjPc6UGviJZbE+Hdvg3ulH4rQjE2dkamMwQ9WPxyV3c415fhPLAPznF8fo2s7umMnM8a2vCp2X/ldAROmqINeLUaFMwwW53oSjVz789fG6UuJbIccruhN/nTeqb9z3rRrgMfPc4dYcKAEr6937zhmGYX06wOMRfHzzibqyXZoVLbGN1m3ThrZ4Hu/zsaJT0+h3NcpsDPKyWH6OB5dPKNY1vKRD21wc0k3/8Jb7z7G2gjCTCWyUZm2geKxckDYaMz1vkUNwwiYM6hL19Sd+bG82VKOUgv1RNBFRHEZ6SnK9nqYv8P83VV8iFiN1ZItzeTy617mNlUI/zKoGwdNNzCE3wqoRKW3zvDh+sPNz0hItMO+NmDsfb8TQqP4dMap/R1PlWXnXaab2p8zCIC/D/HrSYbq3Nbp+TLv89JtL9b8bjtW9bUGu3UGeiX1jvH7zadE/7zOO6p38yWxwVN+O7MkjIlddMLofzh8dvydMK5zc6uQh+tYfe/yS2OvNJjqPnmDngjGJy25VW5pTUzSsOktAKUNzG+Mxcund28efpxaZ+Cbez/X1aybg9Wsm6D95FBxSSVr8bcgwZ4zopXtbO9fJsUOise/RDDeQKrrA5rkLZlpYY1UM6dA59uNx/fGX849yuxhElOVOGtLd0Fpf4flveoOH9m2SawjN8Xp0fRnr1q51JtBIlgVnDtUtVpU3EEjN7zSRcyjjXa629zjVFMdZS5lSl6nfJhHpLCIficiG0P+jjosTkckisk5ESkRkhub1v4jIWhH5TkT+KyIdzZSHjN0w462zFk14oMujF4+Mu93fr/qRoePqZXdAo3e4TLJM9eTF2Dcdkq/keYNDkdKgqERETcK9MHqDvGQyawJAns6ePD2jb6yqxpK5XxvpJTVznmga/AHLgjwrq6rILNhuJ7FL1i9PHux2ESgJZv8iZgD4WCk1GMDHoectiIgXwJMApgAYCuASEQnPRP0IwDCl1FEA1gP4g8nyZD0jN3i/wUxODb4AAGDFtoqY27x57QScdHh3Q8fVK92Tj5jLrhmjJ8/EMZ0SLnu6f35ERlnQEHqXiGwXkeWhf1OdKz2Fe2HCwcPLPxmLIT1jr4fWIcH8rFhyvR5dwU6hjikT4fvtuIGdkypLWDJ3ayO9pE3nsbBaSMWkI5GNx06sfPSLE5Nf5zHSmSOCUz/SNDbNemaDvOkAXgo9fgnA2VG2GQugRCm1SSnVAOC10H5QSn2olPKFtvsagPFmIGrByBfpw3roW7wz7NxRwcVkc3Nin+NHxeYqlnj0LJpuVrKVdDzzfnM8AHtukunQKphGy/kRWc1sQygAPKyUGhn6N8eJQlNQuD4Nz3MaP6iLLeup6g3yEiX5AJrrGbP1pVOjRKyqw84e2TslR7a40ZNn5ZDP3NC81O5FXCMvHZn9TeihlNoJAKH/R+vC6QNgm+Z5aei1SD8B8H6sE4nI1SKyRESW7Nmzx0SRM5uR+4fRuur80f0AuLfuTufCPFfOa9ah3YPBtJmbezrPyQu3ZFbVNbpcEiLHmWoIJXeFE6+Eh22K2HPPDa6Tl/jAnQr1BHmCJ388yvTQRaOXmez5rPpxDupm3ZqGVgaLRubkWSXXa91JlAJW3X06jh3c1bJjknMS/lWKyDwRWRXln95KKNpvW4txgiJyKwAfgH/GOohS6lml1Bil1Jhu3ZxPk5sucgz8cRttkQxvPrBr5k7ATdWYKVbFkA49eWEX/qgfbomRDZQoQ1nREHpDaN76i3GGe7IR1AaF+TlYetskhFfX8YjY0siZl6NvzrLekSbTjuoFs+tqG61aPv7NCUmex5o6bICFiUHa6lyXV08288h5lE70NlqZXfusEb3TMrM6BSUM8pRSk5RSw6L8exfAbhHpBQCh/5dFOUQpgH6a530B7Ag/EZErAZwB4FKlzN6WyMhab0YDhPYFufjlyYfi7JF9cPGP+rV6/+3rzKX+TSQVh2IYYWqdvJjHTPqQjgn/WR/Wowg3cPI2ZRibG0KfBnAIgJEAdgJ4MNoB2Ahqny7t8pvqVY/EXrbGDL31tpEA03SQZ7DJM9nhoWZGvw7SrEvYPsmkN9HMmKJvXcQbTj7UsnNaqbhr4vUal99xqq5jjejX0WRpyE1mh2vOAnBl6PGVAN6Nss1iAINFZKCI5AG4OLQfRGQygN8DOEspVWOyLARj3fRGAwSPR3DzaYdDRKJmG+vWzt4x247M7bLxHPYkXkmDKI8og9nZEKqU2q2U8iulAgCeQ3BoJznMI8DQXu0hIjjmkK6475xhlh5f76gaI6NvjCz8Ho3R7wfJ1kVmRqM8c9nopsftLJyzr7fnSu/nkYqNsR3b6pv+ErnOH6UXs0HeTACnisgGAKeGnkNEeovIHAAIJVa5AcBcAGsAvKGUWh3a/wkARQA+CmUOe8ZkebKekYUwTc0Ri3JzM9CJmBQnbjZ2nsGOJRTsSAJgNfbPUxYz2xCqXfj0HACrbCwrxSAimHPjcU3PLx03AIN09JboZcd9fOKhXS0tYyKSZP1vpl7UJo+zMjGb1Y2nqVYHGpk/meznSqnB1F+FUmofgFOivL4DwFTN8zkAWmUFU0qlZl93GovM5BSPmYolWsBld8CRjsM1F/7h5KbHZkof69rTIcgjymIzAbwhIj8FsBXABUCwIRTA80qpqUopn4iEG0K9AF7UNIT+WURGIjh8cwuAaxwuP8Vg5fd2r0dQVedLvKEBVx5TjCuPKU56f6ONwMk2Gie7382nthw2W5jHeWN6nHt0H5w4RP8yV+zJS2/8q8gwRlLnmvnbjRZc2D100Il4xupT9OrQpulxwIbWPAZ5RKnLgobQy20tICXNyhQC0e7jJx3eDfPXtUyi42SPkPHhmsmpa/Qntd8vT2k5v7uNzmQpemRyXPPQRSN1bztjyhDdSWgoNbEjNsMY+dJ/1ojkMzBFG66pLG3bbC3RjXesycVfgycxf4hYzM6RiIatbEREzou8mxea+DIceR+/Z/qROHVoz6bn4bln2kQjdjNasyTbI3ewIbkgL1IqLoQeyewC9UZMHd4z8UYJXHvCIWk5goqaMcjLYkf27pD0vm4EF4kqkR7tzSd+KT9o31pufhu68tLh/nvyEfqHhhARpYPIRrvnrhyT9LG0jbO5XsEVE4rx43H9m147/cge2DJzWqt0/HYy+uXe7VElTv5sjCoOLe/wz5+Nc+ycT1062lTge2Tv9haWhtzCIC9LXXP8IFP7R+3Js3koSTqtCReNz4YgL9rnkEp+PekwHHMIF1Elosxy4mEtG698/uTv79oASVvPjejXEUf37+hKb4rRM7od5Fl5fqt/3P/75bEAnP8ZJXO6ru3yAQBvX3eMxaUhNzDIy0CdCxOnxv3D1CNMnWNIz6JWr+k5rxnpFuMdEjG0xo5lINtbmFHMauMHdcaNk7guHhFlnnvPHob1f5zS9NzIfPhI2oRp2kDg3esn4r+/mJj0cc0wvMRSmtXP8VgdVId/N5wO1pM5X7jXMZV7Rkk/BnkZ6NWfthwSYMfE2anDe2HRrc35BObedLztNwUnevI23T818UY6fXzziS2e+wOWHbpJUUEuHr14pPUHJiKiuLSp6M3Us9rArsaiOWpmGQ0Q3Jy7leoNwLleD47qm/z0mGQl83NJsdUeyCQGeRkoJ2JB9AfOHW7LeboXBefADejSFodH6dmzWpd29vYUAvYOf/QFbIjyXHTHGUPdLgIRkaumHRVcytDMl/gcuxeZzXDHDe5m6fGs/hbg9Qhm3XCsxUdNLJmGcTtGHJF7eGfJQJF/2Ccebm/iC6eSsPx+8pCY733+25NQYGCBz3j+esEIHHuo9fPIapNsoX38kqMtLkl8t+sM3lgVEFG2u3f6MLz0k7GmerK83hTvikohk47ogQ33NQ+TfeySo/GCiaQ3mSzVezjJfgzyMlDkguiRnVOWT/516EYSbzho/y5tcfuZ1vQsnT+6L/p1bmvqGO9e33oeRW2S6wEZvVE/fNEIQ9t/e/upLZ53tajHlA2CRJTpOhfm4YTDzPUk5WrqZLvntqe7wT3atZj/eNaI3qbmQ0aTKcHRtOG9cNLh3XCrgRwMrLYzC4O8DBQZxEXO29po4bwzwLEYL6H2BbmOnEfP+jMj+nVs9VqyPXmJhlxEtiCfc3RfQ8fvFPGlQm85OayDiMg8bZ1dfrDBxZI478oJA5BnoCfz6uPMZQbXQ1LmW405950zHH+/aiw6tIn+3Sg3ys+d1XpmYZCXgbQVRuRaJ7dNM5dVMxo3Jlx3bNt80zLbimpUstdbXe9Lar9EHa9W//QbLVrqIVNaQ4mI7KSdk/e3y0e7WJKWnFgrbXCPIkNzxyIbJd0yfpBzC5ubFZmnISwdFpAncxjkZSDtsMb3QuuzTB4W7H2yeoIy4E5P3s81rXnaNf+cGJvvT3I9pHGDuiS1X9s8Z5dJGN6nA6YMS9xbGavFT++cPiIiArR5V5xutIzHiV4dj0jqDAcK0RNznj2yj/0FsciA0LIIx0f8bg3sWthqW3bkZRYGeRmoML85yAv3OuXneLFl5jRbsmC60WOjbfnT9qydckQPPHOZ+ZbQN5dsi/letKGYQPS1A7V+euxAXDSmn+GytHN4LTwBMKp/p4TbDesTPZtceE4oWwmJiBLT1md5Fs8vMyOgM8ozMtwykkdSb3ikntKk00iV0QM6Y/kdp7ZqBP/Xz8e3eP6rkw/FPWcdiUcuGulg6chOqXM3Icvk5zj75dqJ9esiaU8Z2Rqlt2KKxxdjyOKgboW4YExwzltkts/iLq1bxSKpJNrJ8hNkDbX6x9+pbR5GDUgc5E04pAu2zJwW9b15vzkBD1040tqCERFloML8HLzy07H4/Lcn2bqMj128JpaA8IigUecissk0ktol1QLTRDq2zWuVoKYwv2UD8tSjemFEv444++j06aWk+BjkZaj5t5yIcQPtHzP+4AUjcM/0YbafJ1L49pqf40HPDgUt3rMiAcvPjxsY87zhc0dmoQzXc0UW97wlyoZqVWVzy2mHYdP9U9G/S1uMHtAJk47okdRx2rfJwaHd26XM3AkiolR33OBu6N/FXFZnq+ltMDXT0CgSu1E10klD7F0OqomurjzbS+G4dAtcKTEGeRlqYNdCvH7NBNvPc97ovhjrQDAZKV6lMvHQLph57nCM1tEbFcut06LPKxORpuGh2mGid5wxtKlHc+Vdp8c8bjKdjIl6Sk88vBuK8s0HliJiSSvy9BFsBSQiSnd66yszQZ6ekUDPXDYKANArokHXLnqCnUwMh9JpCCrpwyCP0lK8m7CI4OKx/fH2dcfYdO4gbTz0k2MHtlqf0CqJKsHC/Bz8eFx/0+dJ9gb/3BXN4/y3zJyWlsONiIiopTpfcsv+GKFnpOcRvdpj0/1TY86HJ6LoGORRWgoHJE5ngtK2bEb2YOoJbpIpr57F6634OXgjorx7ph+pa7/De1ifzIeIiNwV0DdVztQwPz0JugTWjDLRS0+Dpxu5COyWeVdEDPKIDFFNrZt9O7XFW9c2D4m1qycvMviK5pQh3dGjfT4euWhE0ueJrLR6d2yDd66fiFd/Oi7ufhlY1xERZb03rtU35cNMHbDjQF3CbfJznf2qmmnZNeMZpElclynXRM2czc1OZDWHu/IUgO5FBU29XNoJ47p63JIor54b77hBXfDN/00yfvAE5xkZGh4zekAnLP1hf9Prc2863lD5iIgovfTp2EbXdmaqgB8VJ57T3zkFk3hlSr13dP9O2LT3oNvFIJuwJ4/Skrh0h/VA4PUIrphQDADwa4I8PcM3kllCwSnxyv/8FS3X19Gut5iJw1aIiEgfM/Xx8L7R11vVikz9bzc915MpmShbfifJjGuiZuzJo7QkAP583lHIMbEIa1LnjTidtievQM/i36kb4yFeR2S8Oo9BHhFR9sq0GiDTrieuFP5OQuaxJ4/Skghw4Y/64dxRfR09b2RA07N9c0rnMTqWbLBioXa7xBtuGq9lM7zbbdOOsLpIRESU4rKxnS9TrrlFP16GXBM1Y5BHacmte1HkTfDwnkXYMnMaAGDK8F7YdP/UuPsnE+I5FRfm5cS+HcS9+YfeO3+0swE3ERGR1WLVd3+YMgRThvV0tjA2U5ovGIzxMo+pIE9EOovIRyKyIfT/qF0ZIjJZRNaJSImIzIjy/i0iokSkq5nyENkt0Vj9RGmeA0kEbJ3b2T/p/J3rJ8btFfX7Yxc83LuZKXMUiIhIP7fmyNslVl12aPd2zdtkyDWn7tgisoLZnrwZAD5WSg0G8HHoeQsi4gXwJIApAIYCuEREhmre7wfgVABbTZaFsohbN1izZ1UGu+UeuWgk2uXbP3V2ZL+OcSe3t82PPd+Qc/KIiLJXttQAXo+gMFQfj9WRFTTdtMnTkVeA0orZIG86gJdCj18CcHaUbcYCKFFKbVJKNQB4LbRf2MMAfgc2KJABTsQV04b3wjOXjcLmB5qHYHpM/sUYHXqZKvFTfo4Xt06NPudOWj0gIqJMpq2bUqWeskyM6/F6BHeeORSf3HwCenYoiL5Rmrlt2lD8++fj8e3tp6JXB31LZlD6MNtF0EMptRMAlFI7RaR7lG36ANimeV4KYBwAiMhZALYrpVYk6pkRkasBXA0A/fv3N1lsSldt87yoafAjz4GUyk9eOqrVa+0Lck0dM5UTryTSI1SpLb/j1Bavh1s2jfZSEmU6EekM4HUAxQC2ALhQKbU/ynYvAjgDQJlSapjR/Ymc5hGBP3TPz7TRHLGSkPXp2AZFBbkoMvk9IJV0K8pHt6J8t4tBNkn4TVlE5onIqij/pifaN3yIKK8pEWkL4FYAd+g5iFLqWaXUGKXUmG7duuk8NWWahTNOwb9+Ng7nOZzkI5yU5KkogZ8RxnvyUqfyPPOoXlh772R0bNtyjmBejgdTh/dE2zyuyEIUIeGUhpB/AJhsYn8iR2njoHiZmc166SdjbTt2LN4Y9e6gbu2ivk6UqhIGeUqpSUqpYVH+vQtgt4j0AoDQ/8uiHKIUQD/N874AdgA4BMBAACtEZEvo9WUiklmpi8hSHdrm4phDuzq+OGpYZIBjlNGevNQJ8YIBZ6y1AJ+6dHTc7JxEWUrPlAYopT4HUJ7s/kRO6lKYh/9cN7HpuZ09ebkOr4ULmJ+WQZQqzP4qzwJwZejxlQDejbLNYgCDRWSgiOQBuBjALKXUSqVUd6VUsVKqGMFgcJRSapfJMhFZrmf7AnS1IMul0QGNKdSRR0TGtZjSACDalAbT+4vI1SKyRESW7Nmzx1SBiRIZ0a8jhvft0PTczp48N7I2x+rJI0o3ZsdXzQTwhoj8FMHsmBcAgIj0BvC8UmqqUsonIjcAmAvAC+BFpdRqk+clctSsGyZasl6d0XlrXJaAKLWJyDwA0Uag3OpUGZRSzwJ4FgDGjBnDybFkq8haycYYz5WGznDQesZRvfDedzudLwCRRUwFeUqpfQBOifL6DgBTNc/nAJiT4FjFZspCZCezwzTDTjmiB+atiTaqObolP5Rj2lG9LDk3EVlPKTUp1nsisltEeoUSk8Wa0hCP2f2JbJdofVhTx3YhygvPhS8q4DxzSm8ceUzkoEvG9sfZI3vr3n7rvhobS0NENtMzpcHO/YksN6xPcKhml8Jg46fZ4Y1/Of+omO+5OXLS52enOKU3BnlEDgsYqDf8XJaAKJ3NBHCqiGwAcGroOUSkt4g0jW4RkX8DWAjgcBEpDU2BiLk/kZtumjQYALD09uByOmbn5J08JPZUVTcnLPiNVNZEKYh90UQOM1JtsJIhSl8GpjRcYmR/IqfccNKheGJ+SYvXtEv7DO3VHscN7mrqHF3axV6nzc2ePDayUrpjkEfkMCPJV9J58XQiIspsc248ztbjH9W3o63Hj4eNrJTuGOQROcxI3OZmBUdEROSWEf06urYm7pkjeqN7UXMPY2Fe9DViiVIZ5+QROewsA4lXLh3X38aSEBERxWbn8giJvHv9xMQb2eTxS45uEWCuuPM018pClCwGeUQOO/3InujXuY2ubd1IH01ERAQAOS71pKUCpZlBn80/B0pf/K0lcoGeIZsnHd4NvToU2F8YIiKiKLSZM6878RAXS+ICTsmjNMcgj8gFeoK8qcN7tchiRkRE5KRcb3MdFMiyRCTZdbWUiRjkEaUoBnhEROSmHE/z18RsyzZpJBM2USpikEfkAj2VB0M8IiJy06Hd2zU9dnLdOO153cIYj9Idl1AgcoGeBlF25BERkZuOP6wbvppxMrq0y8NT8zc6dl43s3qGMcajdMcgj8gFSkf1wSCPiIjc1rtjMBv0L046BBeM6evIOVMhszR78ijdcbgmkQumDOuVcBvhgE0iIkoR+Tle9O3U1u1iOEZPYyxRKmOQR+SCu846Eof1aJ5zUNwleypOIiKieFKhJ2/cwM5uF4HIFAZ5RC7RDgX5zy8mtno/Beo4IiIix3lS4NvpZB0jbohSWQr8GRFlp3CMV5jnRefCvFbvcwkFIiIid9069Qi3i0CUFAZ5RC5jMEdERNQsVZKeXDlhACYP6+l2MYiSwuyaRC4Jr5UXK8Zj6EdERNkoVdZdv3v6MLeLQJQ09uQRuSRch8UK5nJSYaEgIiIim102vn+L54FUifKI0hiDPCK3hOowT5Rg7icTB+LUoT0cLhAREZHzjj20a4vn/lQZr0mUxjhck8gl0XryxhZ3xhvXTnCjOERERK5o8LcM6h68YIRLJSHKHOzJI3JZKqwHRERE5BafP9Di+Yh+Hd0pCFEGYZBH5JJw4pUrjylufpHxHhERZRmfn8MziazGII/IZb86ZXDTY8Z4RESUbYq7FjY9njqcSxYQWcFUkCcinUXkIxHZEPp/pxjbTRaRdSJSIiIzIt77Zei91SLyZzPlIUonke2WU4f3xHmj+7pSFiIiIreMHdgZAPDQhSPw1KWjXS4NUWYwm3hlBoCPlVIzQ8HbDAC/124gIl4ATwI4FUApgMUiMksp9b2InARgOoCjlFL1ItLdZHmI0sa5R/fFut2VTc9ZsRERUbbK9QqG9GzvdjGIMobZIG86gBNDj18C8CkigjwAYwGUKKU2AYCIvBba73sA1wGYqZSqBwClVJnJ8hCljRsnDU68ERERURbYcN9Ut4tAlFHMzsnroZTaCQCh/0friesDYJvmeWnoNQA4DMBxIvKNiHwmIj+KdSIRuVpElojIkj179pgsNhERERERUWZKGOSJyDwRWRXl33Sd54iWSyI8HSkHQCcA4wH8FsAbItHzySulnlVKjVFKjenWrZvOUxMREbnDwLz1F0WkTERWRbx+l4hsF5HloX/s6iAiIl0SBnlKqUlKqWFR/r0LYLeI9AKA0P+jDbcsBdBP87wvgB2a9/6jghYBCADoauaCiIiIUkR43vpgAB+HnkfzDwCTY7z3sFJqZOjfHBvKSEREGcjscM1ZAK4MPb4SwLtRtlkMYLCIDBSRPAAXh/YDgHcAnAwAInIYgDwAe02WiYiIKBVMR3C+OkL/PzvaRkqpzwGUO1QmIiLKAmaDvJkAThWRDQhmz5wJACLSW0TmAIBSygfgBgBzAawB8IZSanVo/xcBDAoNUXkNwJUqvEI0ERFRetMzbz2RG0Tku9CQzljDPTlnnYiIWjCVXVMptQ/AKVFe3wFgqub5HACthpkopRoAXGamDERERG4RkXkAoq3efKsFh38awL0IzmO/F8CDAH4SuZFS6lkAzwLAmDFj2FBKRESml1AgIiLKWkqpSbHeE5HdItJLKbUzzrz1eMferTnWcwDeS76kRESUTcwO1yQiIqLo9Mxbjymc2CzkHACrYm1LRESkJek4BU5EqgCsc7scLumK7E5Ok83Xn83XDmT39WfztQ9QSqXlujki0gXAGwD6A9gK4AKlVLmI9AbwvFJqami7fwM4EcHPeTeAO5VSL4jIKwBGIjhccwuAa8Jz/OKcM5vrRyC7/1ay+dqB7L7+bL52ILuvP2Ydma5B3hKl1Bi3y+GGbL52ILuvP5uvHcju68/maydjsv13JZuvP5uvHcju68/mawd4/bFwuCYREREREVEGYZBHRERERESUQdI1yHvW7QK4KJuvHcju68/mawey+/qz+drJmGz/Xcnm68/mawey+/qz+doBXn9UaTknj4iIiIiIiKJL1548IiIiIiIiioJBHhERERERUQZJqyBPRCaLyDoRKRGRGW6Xx06JrlVEThSRChFZHvp3hxvldIqIvCgiZSKS0YsBJ7rOLPzc+4nIfBFZIyKrReRGt8tkFz3Xmm2fPxnDOrLF+1nzt5It9SPAOjJSttSRrB+TkzZz8kTEC2A9gFMBlAJYDOASpdT3rhbMBnquVUROBHCLUuoMN8roNBE5HkA1gJeVUsPcLo9dEl1nFn7uvQD0UkotE5EiAEsBnJ2hf/cJrzXbPn/Sj3Vk9taR2VI/AqwjI2VLHcn6MTnp1JM3FkCJUmqTUqoBwGsAprtcJrtk07XqopT6HEC52+WwW7Zcp15KqZ1KqWWhx1UA1gDo426p7JFN10q2yKZ6I5uuNaFsqjey6Vr1yJZ6I1uu02rpFOT1AbBN87wUmfsB673WCSKyQkTeF5EjnSkapYCs/NxFpBjA0QC+cbkotktwrVn5+VNCrCNb499KdsrKzz1b6kjWj/rluF0AAyTKa+kx1tQ4Pde6DMAApVS1iEwF8A6AwXYXjFyXlZ+7iLQD8DaAm5RSlW6Xx04JrjUrP3/ShXVkS/xbyU5Z+blnSx3J+tGYdOrJKwXQT/O8L4AdLpXFbgmvVSlVqZSqDj2eAyBXRLo6V0RyQzZ+7iKSi+BN/Z9Kqf+4XR47JbrWbPz8STfWkRr8W8lO2fi5Z0sdyfrRuHQK8hYDGCwiA0UkD8DFAGa5XCa7JLxWEekpIhJ6PBbBz3Kf4yUlR2Xb5x661hcArFFKPeR2eeyk51qz7fMnQ1hHavBvJTtl2+eeLXUk68fkpM1wTaWUT0RuADAXgBfAi0qp1S4XyxaxrlVErg29/wyA8wFcJyI+ALUALlbpkio1CSLybwAnAugqIqUA7lRKveBuqawX7ToB5ALZ+bkDmAjgcgArRWR56LX/C7XSZZqo1wqgP5C1nz/pxDoye+vIbKkfAdaRUWRLHcn6MQlps4QCERERERERJZZOwzWJiIiIiIgoAQZ5REREREREGYRBHhERERERUQZhkEdERERERJRBGOQRERERERFlEAZ5RC4RkS4isjz0b5eIbA89rhaRp9wuHxERkRtYPxKZxyUUiFKAiNwFoFop9Ve3y0JERJQqWD8SJYc9eUQpRkROFJH3Qo/vEpGXRORDEdkiIueKyJ9FZKWIfCAiuaHtRovIZyKyVETmikgvd6+CiIjIWqwfifRjkEeU+g4BMA3AdACvApivlBoOoBbAtFBF9jiA85VSowG8COA+twpLRETkENaPRDHkuF0AIkrofaVUo4isBOAF8EHo9ZUAigEcDmAYgI9EBKFtdrpQTiIiIiexfiSKgUEeUeqrBwClVEBEGlXzRNoAgn/DAmC1UmqCWwUkIiJyAetHohg4XJMo/a0D0E1EJgCAiOSKyJEul4mIiMhtrB8pazHII0pzSqkGAOcD+JOIrACwHMAxrhaKiIjIZawfKZtxCQUiIiIiIqIMwp48IiIiIiKiDMIgj4iIiIiIKIMwyCMiIiIiIsogDPKIiIiIiIgyCIM8IiIiIiKiDMIgj4iIiIiIKIMwyCMiIiIiIsog/x8kVjLGiEpX3wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3kAAAEWCAYAAADbzV5zAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABy80lEQVR4nO3deZwT5f0H8M832Ytdjl1guY/lRkBAXFEEERUExYr30Vqt2lqv1h5q0WqrVSvV/my1XrXVqvWq9wUKoiAeqCz3Lcu93NfCAnvn+f2RZDfHJJlMZjJJ5vN+vXixSWYmzyS788z3Ob6PKKVAREREREREmcFldwGIiIiIiIjIPAzyiIiIiIiIMgiDPCIiIiIiogzCII+IiIiIiCiDMMgjIiIiIiLKIAzyiIiIiIiIMgiDPCKyjYjMFZGf2l0OIiKiVMM6khLBII8yku/CeEBEcgOeWysilwQ8Hi0iSuO5wyKS5XvcWUT+JSLbfc9vEJHnRWSg7/US3zGmh7z/SyJyT4wyjvPte7tJp51UAeeeZXdZiIhIP9aR1mMdSXZjkEcZR0RKAJwCQAE4N+CleQBODXg8FsAajee+Vko1iEg7AF8DyPcdrxWAEQA+BzAh5G1PEpHRcRb1KgD7ff8TERFZjnUkkTMwyKNMdCWAbwA8j+DKYR68FZTfKQD+ovHcPN/PvwZwCMCPlVLrlVelUuo/Sql/hLznQwDu11tAEckHcBGAmwD0E5HSgNfGiUhFyPabRGS87+cWIvKCrxV2tYjcHri9b9vbRGSZiBwRkWdFpKOIfCQiVSIyW0SKArY/SUS+FpFKEVkqIuMCXpsrIveJyFe+fWeJSPuAzxMAKn0tuKN8+1zjK9cBEZkpIj0DjjdBRNaIyEEReRyA6P3MiIjIFKwjWUeSAzDIo0x0JYCXff8mikhH3/OfAxgsIm1FxAWgFMD/ABQGPHcymi/M4wG8o5Ty6HjPJwD091cyOlwI4DCANwDM9JVZrz8CKAHQG97W0isiHH8CgP4AfgDgIwB3AmgP79/9LwFARLoCmA5v5dsWwK0A3hKR4oBj/RDA1QA6AMjxbQM0V/yFSqmWSqn5InKe730uAFAM4AsAr/reqz2AtwDc5SvHegDxtuwSEVFiWEeyjiQHYJBHGUVExgDoCeB1pdRCeC+SPwQApdQWAFvgbYkcBmCdUqoawFcBz+UB+NZ3uPYAdgYc+1xfS16ViMwKeesaAA9Af0vlVQD+p5RqBPAKgMtFJFvnvpcA+LNS6oBSqgLAYxrb/EMptUsptQ3eSuRbpdRipVQtgHcAHOfb7goAM5RSM5RSHqXUJwDKAJwdcKz/KKW+931WrwMYHqVsPwfwoFJqtVKqAcCfAQz3tVSeDWCVUupNpVQ9gL8j4PMlIiJrsY5swjqSMh6DPMo0VwGYpZTa63v8CrSHo4yF98IOAF8GPPet7yIPAPsAdPbvqJR6XylVCO8QlRyN9/4XgI4i8oNoBRSR7gBOg7cVFQDeg7finKzj/ACgC4CtAY+3amyzK+Dnao3HLX0/9wRwsa9irhSRSgBjEHDeCK5kjgbsq6UngEcDjrUf3uEmXUPLrZRSEcpORETWYB3pxTqSMh6DPMoYItIC3ha8U0Vkp4jshLeyGSYiw3yb+SuwU9BcgX0R8Ny8gEN+CuA83xCVmHwtb/cCuA/Rx9H/GN6/vQ98ZdwAbwXmH45yBN6J7P7zcsM7rMNvB4BuAY+76ylfBFsB/FcpVRjwr0ApNU3HvirC8X4ecrwWSqmvfeVuKquISIJlJyIinVhHGsI6ktIWgzzKJOcBaAQwCN7hEsMBHANvBeWvHObBOwzjVHiHoADAcgC94G05DKzAHgFQBOC/ItJHvFoh+lCM/wLIBTApyjZXwlvRDQ/4dyGAyeLNVvY9gDwRmewbnnKX75h+rwO4Q0SKfPMFbo7yXrG8BOAHIjJRRNwikueb1N4t5p7AHgAeeOc9+D3tK9tgABCRNiJyse+16fDO97hAvCmlfwmgUwJlJyIi/c4D68h4sY6ktMUgjzLJVfCOjd+ilNrp/wfgcQA/EpEspdT3AHYD2KGUqgQA36Tx7wC0hjcdNHzP7wVwErxzCb4EUAVgCbxpom/QKoBv/sAf4Z2gHUZEToJ3QvgTgWVUSr0PoBzA5UqpgwBuBPBvANvgbbUMzCT2J9/jjQBmA3gTQC0MUEptBTAF3onge+BtZbwNOq4NSqmj8M6x+Mo39OQkpdQ78GZje01EDgFYAeAs3/Z7AVwMYBq8w3z6ofkmgoiIrMU6Mk6sIymdiXfILxGlKxG5AcBlSqlTY25MRETkIKwjyanYk0eUZkSks4iMFhGXiAwA8Ft4s4ERERE5GutIIq8suwtARHHLAfBPeOdIVAJ4DcCTdhaIiIgoRbCOJAKHaxIREREREWUUDtckIiIiIiLKIGk5XLN9+/aqpKTE7mIQEZHFFi5cuFcpVRx7SwJYPxIROUm0OjItg7ySkhKUlZXZXQwiIrKYiGy2uwzphPUjEZFzRKsjOVyTiIiIiIgogzDIIyIiIiIiyiAM8oiIiIiIiDIIgzwiIiIiIqIMwiCPiIiIiIgogzDIIyIiIiIiyiAM8oiIiIiIiDIIgzwiIiIiIqIMwiCPMtr+I3W4+Omv7S4GERER2WDtzirc+PJCu4tBlHQM8iijrd1ZhQWbDuBIbYPdRSEihxKRSSKyVkTKRWSqxusiIo/5Xl8mIiMCXisUkTdFZI2IrBaRUcktPVF6+2TVTsxYvtPuYhAlHYM8ymhKKQDAP+dtsLkkROREIuIG8ASAswAMAnC5iAwK2ewsAP18/64D8FTAa48C+FgpNRDAMACrLS80UQbx3QYQOQ6DPMpo/mu74lWeiOwxEkC5UmqDUqoOwGsApoRsMwXAi8rrGwCFItJZRFoDGAvgWQBQStUppSqTWHYiIkpTDPIoo23ZfxQAW/KIyDZdAWwNeFzhe07PNr0B7AHwHxFZLCL/FpGC0DcQketEpExEyvbs2WNu6YmIKC0xyKOMdsfby+0uAhE5m2g8F9rsFGmbLAAjADyllDoOwBEAYXP6lFLPKKVKlVKlxcXFiZaXiIgyAIM8coTahka7i0BEzlQBoHvA424AtuvcpgJAhVLqW9/zb8Ib9BGRThzIQ07FII8c4V9fbLS7CETkTAsA9BORXiKSA+AyAO+HbPM+gCt9WTZPAnBQKbVDKbUTwFYRGeDb7gwAq5JWciIiSltZdheAyCoeD9vviMheSqkGEbkZwEwAbgDPKaVWisj1vtefBjADwNkAygEcBXB1wCF+AeBlX4C4IeQ1IiIiTQzyKGPVezxBj3dX1aBDqzybSkNETqWUmgFvIBf43NMBPysAN0XYdwmAUivLR5TJ/InXtldWo0thC3sLQ5REHK5JGSs0o+apD821pRxERESUuP8t2IK/fLzG0L4vzN9kbmGIUhyDPMpYoUFedT2Tr2QSj0fh4NF6u4tBRERJ8uTc9Xhq7npD+4pmEluizMUgj4jS0ivfbcGwP82yuxhERJQkiYRpLsZ45DAM8ihjKSZOzmg7DlbbXQQiIkoikfgjtUbfsB4DuxKlNQZ5lLGYXDOzhQ7HJSKizGYkTnvs03W+fRnlkbOYEuSJyCQRWSsi5SIyVeP1gSIyX0RqReTWePYlMsrDKCCj8dslonRS1+CBCqiXlFKoa/DgtjeW4p73V6KG88ZjSyBOe2fxNvPKkQTf76qyuwiU5hIO8kTEDeAJAGcBGATgchEZFLLZfgC/BPBXA/sSGcIYL7MFfr/lu6uwYttB+wpDRBRD/7s+wjPzNjQ9fmbeBvS/6yO8sbACz3+9iTf1OiTSF7etMn2G+G/ZdxRn/m0eNu09gp0Ha+wuDqUpM3ryRgIoV0ptUErVAXgNwJTADZRSu5VSCwCEpsKLuS+RUbUNbBXNZIFzLi948muc848vbSwNEVFsD37UnP6/fPdhG0uSnozMyUtHdY3edX7H/XUuLntmvs2loXRlRpDXFcDWgMcVvudM3VdErhORMhEp27Nnj6GCkrP8IE1v+mvqG7GnqtbuYqQ89tQSUTpzSLxiKid+ZDX1HruLQGnKjCBP629O7+2X7n2VUs8opUqVUqXFxcW6C0fOtetQegZK9324Cic8MDto7oaTLK84iDveXhZzO09AZh2ntO4SUeZgIpD4OedS31y/Ha1rwIEjdTaWhdKVGUFeBYDuAY+7AdiehH2JMpJ//P1+h17U312yDa9+tzXmdoEhsHMqfiLKFLxuxc8pgXFgG++hmgYcd98n9hWG0pYZQd4CAP1EpJeI5AC4DMD7SdiXKCP51/RpdOgaEG6dK9Y6tKOTiDJEtUY2zV+8utixozj0YGBMpF/CQZ5SqgHAzQBmAlgN4HWl1EoRuV5ErgcAEekkIhUAfgPgLhGpEJHWkfZNtExE6cwf3DU4LMhraPTg4NF63ZU4F7snonRxpLYh5nNKAR8s3c41Xom1G5kiy4yDKKVmAJgR8tzTAT/vhHcopq59iZzMH+Q5rSfv8Tnl+PvsdbhxXB9d2wc2drNxl4hS2aItB8Key3IFt7P7L2kepeDmVU1TvPOvDx4NTepO5BymLIZOROZxapC37YB3DSOXzkp8zc5DTT8z8QoRpbKjdeFDM0MvW3/xLa/gtGt/PHSO5m/yn683WlMQizl1Tj6Zy5SePCIyz7cb9wNw3nBNP72V+Dcb9jf9zBCPiFJVXYMHP//vwqDnzn38SyyrOBj03PwN+wB4e/JIW7zted8G1BPp5LJnvrG7CJQB2JNHlKKc1prrP1uXjigvdKF7Z31SRJROtK7loQFerO3JGH/g7Fet0aNKlKkY5BGlqAaPMxdA/fvsdTG3cehHQ0RpKN6eOV7fIkt0CYWn5pabVBKi1McgjyhFOa01N9Z90Dcb9uGOt5drvsbhmkSUqhZuDk+6Eg2HawZ7dPY6HPZlIvUP9DC6zEQjP1tyEAZ5lJFmr9pldxES9vDMtRFfu+W1xdhTVZvE0lgv1pIIr5dtxavfbdHclnlXiChVPT6nuffILQJPjAY8BiLN9h+pw99mf48Fm3xz63wX+153GEvKnpflNqtopjusscwGUSIY5FFG+rJ8r91FSNgX64LP4ZVvt6CuwTuO570l2+NuHU53gcN0Qu+BEh3CQ0RklcCrU062K2YQFysIdJIR930CQH/W5Viy3Kl529voURjyx5mGeyiJtKTmbzsRhbnzneVYvq3S7mLYhkOYiMgOL3+7uamBLVFZIjGH4rMnL5yE/G/UK99txpy1uxMtjun8wZ3TpmmQtRjkEaWRC5+aj5XbI2dlS2eHqqMPVdl1qKbp59Bq0P+YFSSlIhGZJCJrRaRcRKZqvC4i8pjv9WUiMiLkdbeILBaRD5NXavL7/TsrcM3zC0w5lgKw82BN1G14GQt317srACQ+NH/r/mrc/+EqE0pkLv9XXt+oYq6R9695G/D32d9bXyhKewzyKCNl8hytyY99aXcRLDF7dfR5lIGN24EZ0nYerGn6vj9bk3ottORsIuIG8ASAswAMAnC5iAwK2ewsAP18/64D8FTI67cAWG1xUTPa5Me+wNcJDONPZArAim3NDXMepXD9SwujbJ1ewzWTVdYt+4+adqwjtam3jIJ/pMrh2gbc9a52gjG/Rz75XlcWaiIGeZRxGho9+G5jei6AGo9MDmRjmbF8Z9PPG/Ye5ow8SmUjAZQrpTYopeoAvAZgSsg2UwC8qLy+AVAoIp0BQES6AZgM4N/JLHSmWbn9EL5ab89c7SMBa7MdrWvEmp1VUbf/1xcbrC6SKbbuP4redxpLgGKU3mt9TX0j3luyTfO1nYei96RqOe+Jr1C2ybr7Cn8j5nlPfIX6xvQJ8im1McijjDNr1S6s3H7I7mKYoqqmHv/5aiPeXFhhd1FsFymoVap5Un5OFi9plHK6Atga8LjC95zebf4O4HYAESeFich1IlImImV79uxJuMCZyu1yoa7Bg1++uhj3f7gK5z3xld1F0vTi/M12F0GXWKMvrCA6Wzfnrt2DW15bYtr7LtlaiXnrrG8k2FZZbfl7kHPwjogAADsOVmNtjNbFeLxRthU3v7LItOPFw6wJ8nYIzaxVVdOAez9YhVvfWGpTiaz1l4/XRMwmpnd+nUcpFOZnAwCy3ezTo5Sj9UsZ+sutuY2InANgt1Iq6vg+pdQzSqlSpVRpcXGx0XJmvM37juBIbQPeX7odH6/ciSVbKzW3S6fhkmZTSmGrzqGR937gndu293DqLeeTbiNdauub71s+ibEEVLqdG9mHQZ7DfbxiB0qmTseoBz/DxL/PM+24byyswIfLdmD9nsOmHVOvTMrCGCvQ+d+CLbYF02Z4au76iENTauojz5sIDAw9CshlDx6lrgoA3QMedwOwXec2owGcKyKb4B3mebqIvGRdUTPbe0u2N0XXFQci95j0vnMGSqZOT06hUkzZ5gM45aE5ce1Tev9si0oTLjC+mfT3eXj2y422vLfZbn8rMxtyyV68M3K4u99baclxXb6r4Y7K+Me+JyqdG2FD49NY8ep7S7bjw2U7cCQNF1GN1Vp+tC5KkBfwc0NjcwuoUsBDH6/BRU9/nWjxiMyyAEA/EeklIjkALgPwfsg27wO40pdl8yQAB5VSO5RSdyiluimlSnz7faaUuiKppXew73eZN7olXVTV1BvaL2kjaAIirTU7qzDve+uHJ0fq8QXM6/XdvC927+mq7YeglIpaNxIFYpDncHuqrBlm4Z8jpcJGJVkvk3ryYp2L/+V3I0wwT2UNvspRQWnOOaxr1HfTcO0LZU2BvUcpfP79HpRtctZC8ZS6lFINAG4GMBPeDJmvK6VWisj1InK9b7MZADYAKAfwLwA32lJYBxj5gP5ep0WbnXcd8RiM1YwGh3ELbQiNsJmZvW7R5m72vnMGPk3S3MSzH/sCK7ZlRr4BSo4suwtAqeeq577DrWcOwLHd2hg+hn/MuB3x1rKKyuS/qUlCPy69AaukYX5J/7kpBXyxLrw11t9Dd7i2AV8GTHiPNIfPf6wW2W6TS0qUGKXUDHgDucDnng74WQG4KcYx5gKYa0HxMl7gGpsNcfS8ZE5zoX56F2LfvO9I0GO9CVES0aCz4Q9ITnn8Nu49Ensjk9QbjcLJkdiTR2E+/34PPlwWOmUkPv6gI9mV5P4jdXjpmy1JflfzhAYw90+PvDTWDTHWWkp1gTdbWvcV/rl6ry/YiutfWtg0JDXaPYhHKeT5grySqdOZqYyIcOKfP434WrSheKGiNTBFU9qzyNB+ybLvcC1Kpk5H+e6qpuGH63ZVRT3fUx+ea1l5qmrqsWRrJRZvCe5JvfWNpSmZdOTzJAwZJTKCQR5p+ue8xNbpae7JS26Yd7Qu/eamAd7P6fY3l4YFxdEW9w5skE7Fii8Wf1KZSL8iM1d618LzZ8xcWuFdUNijVMR9lApeRmHnweTPCSWi9LFkS+QhmaHXGaPTr1J9BsEffHPzF22ubOrJm/C3eVi0pVL3MUbc94lp5Xns03U474mvwuaeff79HiwIGYqfzHuMSPXsF0lYWsEv1X+XKLUwyKMgetPWx+IfKrGtshoXP/01Hvp4jSnHzVT1jQqvl1XEnUBF77y1VNQYMCdPy8Mz1wIIH3azekdV0D6dWuc1/aygkONuvqzFM7yHiJwnWo0Xem0yWj96Unzg54GjdQCAbzbsC8o6muxGWj//KI7Qtz9wVP+8PyvaPY1Mi7j8mW80G59P/7+5WBgw51P/8NLU/l2i1MIgj4L4b6wTVetLf//7d1ZgwaYDSVs0NV1buep9wUhlHJUYgKBKIt34b5jmfb83YrVVMnV62NyPpz9fH/R4dN92TT97PEB2QE9ePPNviMh5/Gu9+a3e0ZzY4vO1e3D+k81JN4wm9dq4J/45W8lK5gE0B09vL96GaR81N8jm2Lw0TSJJ1JI9uuXfX2iPfpq/YZ/miJINe47gwqe+jrshku2WFA8GeRRk1Q5zMjd9u3F/0ONk3WvHqhRmx1hk1C7+IM9oUJKqozW3VVaHBWl+/iDv1e+iz6H81xfh6yAFtqh6VHOLuwKCevLSuaeTiJLjrYDsvmc9+kXTz5+t2Y3FAUMWjfbkVVbHn3kydFiilaojTHOIN8ir9PUImiWR24ZrXygzrRxa3llcgUc++b7pcbR5eVmuyJ/j9OU7AOivw6/+z3c6tyRikEch/EsqdGyda/gYFQfC13sp352cRdHP/Fv0Bd21sjgm4rM1u0xZJ8cfjLz0zeaEj5VKpjz+VcQJ+oFZ3OIdFhQ6jMq/+6rth4IWUW+IsNA6EZHfb9/QtxB1tAXUzZblSl7T3aYIa7RN+vsXms9HMvKByAlujLjqudQNaP7xWTke+3Rd1G38jQLRehXjraOOcI08igODPAriH6pyyEDLo9+ctfZlmqqNsSCr2WmVr3m+DGt2Jr5grn8h2ee/3pTwsVJJtEQ4/uB46/6j+HDZjriOGxgTNnpU07H+Nvv7ppZR/2tERGaY+PfojYhmcgcEeeW7q5K3Dl0CMnHkxP0frtJ8PnDESCT+ej1aPZSOSdMofTDII02xgqVo7n53hYklSX1mLPieaDCSqhWFf/hstBuUDfGuMRRyru8v3Y7VEQLtROZ0EBHZJbAnb/wj83Crzt5GpzF6iY8UvIX6btN+zef3Hq4NeqxV1/hHq0SbhuGvu82aKkMUiEEeabKiA+TF+ZvMP2icUjcYsrsE1vAHr6u2H8KWkCFBRgOweIZ2NngUM2wSkS6jp31mdxGauN3BldXMldbMJ/9uo3YQky6+LN8bth7q97tij65ZFGXpDD32Hg6ef6hVLfnnKN7z/krUNjQPs3x4ZnNym1kGv9ei/GxD+5GzMMijpFmx7aDdRTB1jtZS3yK6a00YrpmpPU7+VNiXPvMNfvTsN03Pn/v4l03r3sWr0aN0LzVx34erMOzeWYbeh4ico77RExYs2Jmd152kFskXTJ4iMPSemaYeT483yrYGPV6vIwdAtGQogQK/Bf9XUqcx0kmrCr/2eW/yly/L9zbN37vj7eV4Yk5zhuiPVuzUVY5Q8SwnQc7FII+SJhXimP+amNjkTV9Gtt+8nvgwmkTXIzKyfo+Vdh0KTxm9dX/zDdSyioNYEsdCu4EaPQodA9bGi2ZPVS0nqhM52F91LgvU7/cfWVyS+LiSFOQFzmE2w6Ga+NZ6NUNo/efWk7TGwMfr30VrSYT5G/ahZOp0VAfUN2sDehT9gd3biyrC9jWqqqY+KNEYUSgGeQ5mdqZJAHj+q/B0936ZOiTRDKkQAJul0aNw4p+1s6wFDqPJz3EbPr4ZyW6IKPPZnczKjOzLqUhP4pF4bN1/FPuPmLMEQ7aOshkJof/9pff+JlonYKQ5fH5m/jYce88sPDB9tYlHpEzDIM/BfvysdnriHLcYXgT1nSXbI75mRoKSVGJmC1rC9wEp1JHnH3qqdXPz1Nz1Yc/Fa4dGKyoRUahGj8JhnUO7LStDgi14/e9KrR5GAKhtaIyaSdPIyJRTHppjeMmE0E5PXT15Bg5+0Jd1vKY+8rnf+/7KiK+t2HZQc6hnInZqjJoh8mOQR2HqGo0nq1i1PfI8q7cXbTNapJT0xkLzhl0kOifv9jeXmVSSxL22wDs/ItbNjdGgn714RKRHfQokXWr0KCQSc5gdFAQyWs/f+Xb0DNpGk7mEZqw0Ss8ag4mMhr3m+QURX/Nni77r3eVhr/3sRe0F2v3z+43IyzY2IoacwZQgT0QmichaESkXkakar4uIPOZ7fZmIjAh4bZOILBeRJSKi/RdASedRxoaZ1Nu4+HSshUlTmRmJV1JlHSX/Ehob9mgvjeC/AXj1u62arxMRZQqPUobm12ntoidrZDzW6UhQcp/GUgPbKrUXT/c7anCUiz+g1ZrzFk3oR5Vl4lBSrW9uy/7o5w8AL32zJey5SKNQVkRpHI8ln0EeRZHwX4KIuAE8AeAsAIMAXC4ig0I2OwtAP9+/6wA8FfL6aUqp4Uqp0kTLQ+ZwSeLDTJLt5W/NS6qSbGZ81D/5T+TWRTtEWjh4wx7vjYVZ8y+IiJIl3sbPRo8ybemeb01e7mD9nthBnlavXKw5b0aTxtT4lhn4eKWxjJN+pg7XNODT1fEti5BI4rRCLqVAUZjR3DESQLlSaoNSqg7AawCmhGwzBcCLyusbAIUi0tmE9yaLCCThBbqTLVkppwHgUEiv2ZqdiS1kakZPnn++QKqzMy05ETmHFUvTxHv98niMT5kOXFsNAFqY3Gtz8yuLY26j9RnGSrpiNMYyWjUEVv3VdY34eIV5GUON3FZc+0LyBqXlsiePojAjyOsKIHDcVYXvOb3bKACzRGShiFxnQnnIBCJIuyBPkhjkhbbm1jck9lmZcS9SU9+ouXRBMsU7zIaIyCpWDEaJdy6xRylDddP901djwF0fBz2XazAhWiK07gNcMaI4oz15yuA9R+DnO335Dvzri8hZvomcxIwrhtZfc+hfarRtRiulRsA7pPMmERmr+SYi14lImYiU7dljfup/CiaSuT0ui7ccSPgYoWPrda6rGpEZLc4VB6ojLl2QLCc9GP39Gz0Kd70bfdI+EZEZrOjJ88SZq8TMaQ92DEPUKn6HVrnWvJcZx9D5eRv9WqxIhJNo+/S0j9awgZU0mRHkVQDoHvC4G4DQPPoRt1FK+f/fDeAdeId/hlFKPaOUKlVKlRYXF5tQbIpGRDJ2fR+tCdHxevCjNUGPE618M/SjDmNlpjgiokBWXFeH3TsrzjKYNyfPjplma3dVYVtlddBzm/dpJ9XyMzoKKBXTAIR+5v/8PPFlgMwkAJ7+fD3eX5pZ2cvJHGYEeQsA9BORXiKSA+AyAO+HbPM+gCt9WTZPAnBQKbVDRApEpBUAiEgBgDMBsJk/BQis6clLhSGgofMcjKgPCVYSnQ9oZF2hdJRuyXyIKI1ZcLmJtj6clkTm5KWKRZu9o1/eWVyBfr+fgS/L90Xd3mgPqtFldR6euTbgGDrfy+DvRve2+cZ2jGLVduNz+v2ncaja3vUgKTUlHOQppRoA3AxgJoDVAF5XSq0UketF5HrfZjMAbABQDuBfAG70Pd8RwJcishTAdwCmK6WCB6GTJWK1xInEf6HWE6j87i3713OLtpCpXqHrL939XmJtEykQ+yZFpvYOE1Hq+cP79rcZexu2zAnzkjjtPEitr1FzyZbKmMsk5WW7sKfK2Hp3plQPOo+x70itocbVFjnmJzrZsDd2ltNI/A3nRgNkymymzOJVSs1QSvVXSvVRSj3ge+5ppdTTvp+VUuom3+vHKqXKfM9vUEoN8/0b7N+XrHe4Nnqrj5HsmnpS4m/aGz24TAYzFsgNbc39ZkNiqa0zoSdPTwBnV0/ez//LJTiJnOLWN5biy3V78d6S0JkjyecxcQkFu/oE/aNfXpgfe5mimnoPbnvTYGOuwfV5gfjr0PV7juCDZbGzcIYmzWmwYC3g2gQanv2f18crduLvs783q0iUIZKfqolSQqw5ZEayaz7/9aaY26R/KONl9rBTMw9nV8D47y83xNxm7lp7kibNXBnfukVEZhKRSSKyVkTKRWSqxusiIo/5Xl8mIiN8z3cXkTkislpEVorILckvffp5c2EF3ly4NfaGSWBm8pfrX1qIrToW4jZbXYPHlGkOsSgozI5zjTk//3p+8fRobdwTf6NzQ7yZd3Q4Umf8s21U3vKs33MEf5+9zqwiUYZgkOdQseaQCeKvnFbqGFeeChk7Exka4Zcfx5CNsx79AgePRl/DzszATE/rpBW2HaiOvZFNXAKU766yuxjkQCLiBvAEvBmkBwG4XEQGhWx2FoB+vn/XAXjK93wDgN8qpY4BcBK8GahD9yUN9tc0Xo0eZWr/24GjsUfMmO3JuevDlnNIVF2DB28urAh6TqnmoaHxijWMVMvfDPR83fLakrj3ieVQAmvchsacdjQCUOpikOdQsda5AeLvrTpaF3vir5XZFfUGSlv3Jx6MnD6wY9hza3eGBxEej8LqHYewKWQOZF2DBw0BQz7NjH23xJhvaZUlFQdteV893C7B+EfmoaomPRaMp4wyEkC5b3pCHYDXAEwJ2WYKgBd9Uxu+AVAoIp2VUjuUUosAQClVBe+899B1aElDqoyA95g3JQ+A8TXoEmF0jh0ALNy8X3MI5qItB3DrG0uDnlMwHpz7P5ZU+d7jkcjIoNCG81MempNocSiDMMhzqNcXxBjKIvEHHnoCuDoLh3wks5NQa0jIxL/PC3vuNd/nfO0LwXPCxj40B7/635Kmx2bOVUvmovCBlm6ttOV99fD/bqRCdldynK4AAi+4FQgP1GJuIyIlAI4D8G3oG3Ad2XBWrJFnRIPHAzExystyp36uzt7tC5p+vvCp+fh6fXg2Tv/C7gs3B69be8triw29p/9Tifdb33s4egCbjE975yHja9zF6vmsbWgMalAmZ2GQ51D/nBd9/pRA4q4k9dw/GxlSoVciF8pAJVOnx0xMo3dyeGW1d2hNaEWy81ANlgX0fJl5Q2LHgrmpzh/c/Z4LsVPyaf1Bhv7BR91GRFoCeAvAr5RSYePiuY5suNQI8cxP1PFGWUXsjXSotHDY54aQBGuhjaLVdY04/8mvAQAXPvV18LaGu/KM7bb7kPFeylQQawTVuIfn4uZXjAXOlP4Y5FFEViyhsGX/UdTUWz+BO1GxhvX9dZa+sfyNARX8M/PWo+JA83j5wM/XzKUFEl2zL5NNt2m+IjlaBYDuAY+7AQhN+xhxGxHJhjfAe1kp9baF5cwoVi7XEs+x6xs9pi598OyXG005TiJDMON1JKDRtKa+EVs05o0l+hEZHcYaK1FLqlenRyI0SC/dWonvNu7HjoM1WL4tdadSkLUY5FFE8SaR0lvtPTB9ddxlSTajrYklU6djW2XznL/qgID2zzPWYMxf5jQlYQls4TXzfkTPfEsiSpoFAPqJSC8RyQFwGYD3Q7Z5H8CVviybJwE4qJTaId6x188CWK2UeiS5xU5vGwxkTtTrUBxze1N1iHgyRnxs99WF17+0CH3unIGNe49g4N0f46cvLgjbNtFgqmm4Zpwfd6ztzRxqawWtzJxLtlZiyhNf4ZJ/zgdgTUZQSg8M8kiTkeyay3Qm3thx0JxhlYmwsjfxrYCMYU/OXR/2+tKKSgDBw0vNvBGwI8bbbFOyF6JUp5RqAHAzgJnwJk55XSm1UkSuF5HrfZvNALABQDmAfwG40ff8aAA/BnC6iCzx/Ts7uWeQnnq2y7fs2Ov36M/QbOUUhUQkI9P1wzPXNv3c6FH4ZNVOANrJz8yaS14dZ92eDiOLolnuu58I9MgnwSONrFjbj9JDlt0FoBQl1k1cT8Z6O7E8PHMt7j4ncibyRM68qCAn6uvZ7vC2FTOXUNA7bGXJ1ko8Oaccs1btwuUje+DBC441/J5XPBuWC4KIfJRSM+AN5AKfezrgZwXgJo39voRdK2Cnuf1HrJtzVhPH4tVW9KI0elTCPXFWZroGgGtfWIBPV+8Oei40+AiU8HBN3+dxMM65hhc9PR+bpk2O+Ho86+7Z4XBt+P1UaG6eevbkORZ78igiq4aZrNp+yPRU9nPX7o69UYADR+tQebQOByOsT5NI0PWveRuiztn4w3vhyT++9S3kaobC/Gxd2533xFeYtcq78Oyr321JqEUz1Ye0EJEz+OstK2/NI82D0mJFj9nL325O+BgNHoVWeda184cGeED04Nis4ZqVCaw5lylCe0VTJNEs2YBBngPpXSzTqtEc+47U4Yz/+9zUY36xbm98Oyhv1qlLfWPWw15O4Ny37D+Kyup6bNyrPYRx3e7goT4HjtTh+a83GX/DEHnZ+hdqD/TO4m2G35PTAIkoFdT70sVbeU06qjEPKpJGC4bK+ed1JyKeQDUZEm0oFBFs2HM45pII8TpgwmedbJ+tCQ6wq2pS67um5OFwTQf666y1sTeCuUMIQ+1OYmYvLW/7AhorJ8Uf0tGiuONgNUY9+Jmp76vne9PaJpGv2661+YxQSqVVeYlIvzpfkGfl6IIjMdLWB2rwKNN7UvaYEMjsPVxraQbSZGqVl4Wqmvqw5RjMoJUJlChdsCfPgd5bEpq9O1xVTQPmxds7ZiOjlWjoWHV/8JNopexR+kbyr99tfsKSf3xWHnObpz8PXycxkbhHQv5PZfG0whNRelFJmH50OI6eEW9DornB1IvzNzf1WBpV35gas80Kcr0jT+oSPB+rgrFMWZJol0nrCFN6YZBHET2tkRkyVRlNEhM6R8B/mMYYx+vcJi/q65+u3oWV22NnG83NNv9PcOX2sLWSw/zl4zVhz93x9nLjb5pG9SAzjRFlLn9dEOsanoh41h1r1NngF69EE6OlSi9eo0c1LbVglMejsPewNYl2alIgUZwZkpFNlVIPgzyK6Icn9rC7CLqZlQnUf5xYx4v1+u/eWo7fvxOeYCVUuqdv9kunid3MNEaUufyXohYG5ybr8eGyHfh2wz4Mu3dWzG0bPR5LssAkehlrVMrSKRl6mRF7HKlrxGOfrjO8f7R1D0/tX2z4uKkkjdphyUQM8khTtlvQrahF0t/3cG0DSqZOj3s/s1qp/IeJ1cpp1lw+qya/b9l3FAcsTCEeqtKXttr+W4bYqjlckyhj+Rvgviy3drrB1c8viJidOVCDRcMiE+2ptGKuoBEej7J9wfjy3ZHXPTy+R1FGBEh2f8ZkDwZ5pMmjlOWLuL5etjXsOT3JSrSYNfTEf4Pw1Ofeoaorth0MW0+oqqbetKEhlRZl7hr78Bz8/KWFlhw71O6qmrTKQGZ2ZlciSh1Wre8aSu86cx5lLJhySfQMoUZv2j0ehZKp0+HxpMacPO+9hr2jK6J9lqnwGZkhniHGlDkY5DmM7orJAzRYfOG9/c1lYc/VGlyg1axWKn9l/PYib/bNc/7xJV79bkvQNmYGZlYmAYmUNvmj5TtMfZ/aOBYGTgWJTvAnohSWpLtyvaNHGjzK0ILaHhV9KOMNLy3Ehj2Re6Ai8fcA1jd6Uma45q5D9mbbjjZPW6nMCPRufHkRVu8Inq8f+pgyD4M8h1m7s0rXdgqIu3UtNyvxXyd/EHqwuh4VB/RnyzJrkr1WZVwbMvHaZeICTEaDWj0iVeDRsqsaqfSz3Ok3mGXp1kq7i0BEFkj2qLRYgVajRcMiv16/D6fHOSphxbaDmOFr5Ptsze6UCV7szvwYrfc3Nfo7zXHWo1/g3H98GfS4Ksp8REp/DPIc5gePfxl7I594e5naFeTEW5ww/sBy2L2zMOYvc3TvZ95wzeS9FwAsq6g07Vh+BTnehANrdlZpljU/N3JCAiO9XFauR2WVKU98ZXcRiMgCyRqu6XfpM99Eff0P763UPYLGiKqaevzo39/o6tW74+3luOW1JQC8QWIKdOShZW4Wyjbvt7UMTso8uWzbQTz08RoM+sPHAIBTH55rb4HIUgzyKKJIw/0iMaM3zejFtk2L7ITfGwi+QTgYYVimmRXCRyt2mnYsv8BlIfQkBghkJCmJlanKiYjikewgb09V5KGG/tesLNGNLy/CV+X78FX5XvzilUX4ncY0CL/QQSjJ/qy0CIDvNtob5FUcOBox4VsKfESme3Lu+qZG/P1JTNBGyccgjyJ6c1EFfvTv6K2Uh2rqccb/zQVgzjAZo/MAX5i/2fB7Bs7nC1xI939l3rl4W/dXY1lFZdPcPLsniccSuHZrvF/JVc99F/f7bdxj/oLuRERG2HVTXtvQiNveWIrJj33R9Fwy6oov1nmziD40cy0+WLYD7yzeFnFbCVnYOxU6sKpqG/D9rvjnFppp015vHbZVY0H1FPiILBdvYzClDwZ5FFGjR+Gr8n1Rt6nYX431vpt8I8lPQueAWZ3RU8vDM9c2/RzYsunPoPnfbzbjkn/Ob1osvL7Rg5a5WQm/b6u8xI+hJSdgbuS+w96WZI9HoaqmHlU19U1JZbQsrTiIM/8W3zyPK5791lhBiYhMZleQ9+yXG/HGwgqs3H4IHyzdjiuf+87SOdeh/CNvos0hM3E6eUaZvsw7T3H26l3hc9QysSsvhJ71Hik9McijhPgDu3W7tOd/xTL0nuCLS0PICq8lU6fj7UUVlmYBC5z0/dI3zT2Cz8zb0PRz4BDI+kYFSeHKMjDYnvC3edi49whenL8Jx94zK+rQIj+7W1WTJdV7ZIkofnYNQTxU3Ty94RevLsa87/fgtL/OTXo5ojWUulKw4spKgchz+0HvPcC9H6zCsffMwqItB5pey/wQz8vfIEyZhUFehjj70S+wcnvzOiizV+0Ku4m1YjHMel9Q9s3G/YYq16qQxcC1Uhn/5vWlmL8+eo9iIt5ZvA2/fHUxauobMff7PTG3r6lP7cW0Q1uPT/vrXNzzwSoA+hOreDwqZtC+93CtJYljkuXNhRV2F4GITGbHTfl9H66yNLlKvJZVVDbVU+8u3oY/vrcCh2sbULb5QIw9ky8Vk57UNXhQ3+ixLDNqKvqyfC/Kd+vLvk7pw5rxYpR0q3Ycwrcb9mNwlzaYs2Y3fvpiGZ69qhRnHNOxaRujwYnHoyIuG+APyuoaPKYk4KiorNZ8/oiF68kBwPtLt+P0gR10ZRS9LEY2Nb3iTWxjhk179S1LccWz36KmvhFv3zha83WlFErvn21m0ZLujreX47ITuofNUyGi9OVRCgW5bhypTV5j3LNfbsQPT+yh+VqWS5IeyJz7+FeYMrwLHr3sODz9+Xqs2VmV1KGj6c7tEkx45HNs2qd/Gad058+6esdZA3Hd2N6sFzMEg7wM4u9Je71sK4DwwGinwbVojtY3RpyD5l9DTikFj8E65KPlO3DWsZ0BADURgqzGgINv3X8US7ZW4hevLsaN4/rgybnrjb1xiOr6xqiLg7bIduO3ry815b3scv1LC3Vt93WUntPDtQ0pMcTGDLUNHuRlR15SgojSi10LfL/y7RbN560YQaPHe0u246vyvU1zy19bsNWWcqSj1TsOOSrAC/TgR2twSr9iDOrS2u6ikAk4XDOD/OOzcgDe9dEA4JevLkbvO6bjpy+U4cQ/z8YZcS6c6rdkSyVKpk7H/xaEV2L+1tIDR+vC5tPpdcPLi5p+jjScMHAu2diH5+AXry4GANMCPABwB7RcaTViVdc34q1Fzh7it3RrJYb8cSZ+/l99wWKq28t5CERJN2ftbs1MhmZQKrXW7rRztJ8/wHMCo9+41n5/eG9lIkVJews22bukRbKt2HYw9kZA0xBeALrWpUwFDPIyyMHqepRMnY6Ne5tT2nuUN2PUrkPGb2anfbwaAPC7t5ZDKYXdVc09gv41Vp6Ysz6hzJiHaxtQebQOtRGGlN793kq8u3gbDlbXWzZG/qMVO5p+dso4/HjtO+L9Pfpcx9zFdDDmL3Oaft5eWc1U0uQYjR6FT1btCnve41FYtOUAPlq+Q2MvbzKsr8r36jr+4dqGoLlq7y3ZBo9H4er/LMAf32++kT5UU48Dca7XdfBofVOg+Jv/LWla1zQFp3hREhj92vnrEu6P76/Ews378fhn67B+z2HsPVyLR2atxeZ9wcslNTR6cLg28rSTg0frm0Z7xaOuwYOjdeHH3bzvCB6dvQ67D9XgwJG6iPNgy3dXodGj8N6SbRHXPwSAOWt242B1Pc75x5e6etxPuH82/vj+CmyrrMbpAZ0mCzcfwBu+EXSl938S1IC142B1xNEF1XWN2F5Z3XRPXdvQGPR5/eyFMjw4Y3XQPq98u6Vp6tVFT32N2RrX8EAcrkkxrdjWPITxiTnl+Ous73Hx8d3w8MXDTEu88avXFmP26t3Rt/nfElPeK5I5azMjcDHTyu0H8cD01XjlZycByMzg98rnvsOzV5Xi5GmfYWz/Yrx4zUi7i0Rk2K5DNbj/w1UoyM3CT04uQaNSqKlvxEVPzce/rypFn+KWyM1y4bFP1+HRT9dh1Z8mIj+n+VZgSUUlLnjyawDAezeNRr+OLfF1+T5MfXt5U8/3M/M24LkvN+L+84egbUEOlAJueW0x/vCDweha2AK3vLYY7y3Z3nTMC0d0axoF4b8x8ycGu+nlRZi+fAe6FbVAxYFqHN+jCHf/YBCWbDmAzoUt0Kl1HqY88RVm/mosFBQq9ldje2U1/uALEju1ycPOgzU4f0RXDO1WmBILfFPmEDgzELzwqfkAgL/O+h692hdg494jeOyzcvx6fH/MXLkTF4zoin9/sRE7D9U0/e3+38XD8Ns3vFNaVtw7Eac/Mhf7DtfhhJIiAMDfLh2OroUt8M2G/ZizdjeembcBndvk4YHzh+DpzzfgzEEdcd5xXfHXmWvx2oKt+OaOM9C6RRY8Cth/uA4Pz1yLD5ftwKItB/D593vwoxN74CcnlwDwJvGrqWvEmwsr8HbIWpF3vL0Mr363Fd/ffxZyslw4cKQOSyoqcfXzC/CbCf0BAAPv/ghP/eh4nDqgGAJvzoT8XDd2HaxFh9a53o6I6nq89M0WTBrsnV50wZNf438/PwkXPuW9Xk4e2hl7D9fhw2U7cN3Y3nAJMOrBzwAA/712JE7pVxxUroue/hort3vvr5+9qhRPzlmPhVsOYFi3Nli+7aC3wWo1UFVTjx+e2BPr9xzGne8sx8Hqepzavxhlmw/g/z75HhBXxA47sWv8eiJKS0tVWVmZ3cVIKdFaK6wy45en4OyAhV8p80wY1BGfrNqFi0Z0xZ8vGIoz/5bZk9GHdWuDF689EXUNHhS3yrW7OARARBYqpUrtLke6yO3cT3W+6u+ar502oDjuxqzAAC1dtMzNitrDQETWuqS0G14vS6/rRqh2BTnYF+cIg1iuHl2CWSt3YVuEJING7HjhV6jdsU5zxDKDvAzg8Sj0vnOG3cUgyihld41HXYMHXQpb2F0UR2OQF5/W3Qeotj96xO5iEBFREkQL8kyZkycik0RkrYiUi8hUjddFRB7zvb5MREbo3TdTBU7g9FNKNc0riAfnERGZr/T+2Th52meYu3Z3WKubXX9zqbQWF+mXzDqyTYvs8PfXW06d21kpFcpARNHZnWE71a4T0cpjZllDkwIKEDWlcMJz8kTEDeAJABMAVABYICLvK6VWBWx2FoB+vn8nAngKwIk69w1zpLYBN728CMWtctG1sAVmr96FG3yp9DfuPYKJgzviB0O7YPm2g8hyCe75YBWGdy9Efo67KTV8r/YFOGdoZ/QuLkBtvQcvfL0JB2vqkZvlxqg+7TC8eyGWbq3E9spq/GR0L0z7aDW27q+G2yU4/7iuUEphacVB5Oe40aNtPnYeqsGanVXYebAGV47qiY6t81BV04DN+46gbNMBHKlrwEm92+GzNcHzzvp2aIm8bBduGte3Kcvkn6YMRr8OrbBi20FU1dTj83V7sXRrJa44sQemHNcV9Y0efFW+F7X1HuyuqkVhfjZaZLtRbfEi3W6XhAWmTh2vTs7xk/8sCHo8pm97fKmReGJM3/YY2KkVllZUYt+ROlQcqEb3ohZYv+cITunXHgU5WehS2AKV1XX4wdAumLt2N16Yvxktc7Nw5aieqKppQP+OLdHoUdh5qBbvLtmG0p5FGDegAxZtOYBWeVn45+cbcEq/9jixV1sM616IldsPYdpHawAA/Tq0xIgeRbjkhG545dutGNWnHW59Yyme/NEIbNx7BIeq69GhdR72H6nFvsN1+GLdXnRv2wJH6xpx7rAuWLOzClkuQY92+Xjo47W46PhuOLFXW1QcqMajn65D/44tcc3oXpi1ahc8SuGyE7pjWcVBbNx7BEfrGnHm4I7IdrtQ0q4Aj88pR2nPInQpbIFtB6pxpK4Bz8zbgEmDO2HnoRrcfc4g5Ga5UHHgKCqP1qNbUT5cArRukY1Xv9sCj1KYvXo3fjdpIFwFRcVhH3YasaOODKX3Gp0K1/JUKAMRRWf3Ivapdp2IVh4zyxoazsU6dsLDNUVkFIB7lFITfY/v8BZEPRiwzT8BzFVKvep7vBbAOAAlsfbVEm3OgVNpBWBERIHsWJg5UdGGoqSDZNeR3fsPUe4L/mLJuRARUWrZ8fwtqnZnuebITDOGa3YFELjKZoXvOT3b6NkXACAi14lImYiUeY7qW9PCSRjgEVEs6RbgAbBvdWvzWF5HBtaPRw4eMKXQRESU3swI8rRaWEMr5Ujb6NnX+6RSzyilSpVSpa78NnEWMfNlu9O2oZuIksQdOqA/HUg6FjqI5XVkYP1Y0KbIQBGJiCjTmBHkVQDoHvC4G4DtOrfRs2+Yzm3yAAB52S4M6+YN+H4wrEvQNlec1AN9O7REp9Z5EY+T7RaMG1CMsf2Dp3zkZbswYVDHpsf+dTj82hbk4JjOrcKO5VeYn43h3QvRMlf/lMdrRje/x+i+7XBpaXcM69YG/Tu2DNrujIEd8JOTS9C5TR7at8wBoD3RnoiskZsV+bKZn+OO+Fq2WzCgUyu0ysvCD4Z1QdeArJ2d2uShV/sCjOrdDmP6tkf3oubXxvRtj66FLTCwU/M1p0thHi4t7d60/lCg3/rW/RnTt33T/8f1KESf4gJcckJ39OvQMmyfHm3zm8o/uEvrpudH+PbzO7ar93qbk+XCpaXdm65BADCwUyuU9izCbRMHAPBeJ0tLilCUH359+uUZ/XDX5GNw5qCOOKZza1x/ah/8ZkJ//OL0vnAHTOgfWdIWntqj6d41lfQ6koiIyIw5eVkAvgdwBoBtABYA+KFSamXANpMB3AzgbHgnlT+mlBqpZ18tiSyh0OhRQTcRdtl3uBa52e6gQLCuwYNN+46gf8dWUfYMt/dwLUrvn212EYkI3nVt+ndshctH9mh6btX2QxjUpXXTSMJonU16ttGr4sBRdCvKD3rO41E4Wt+o2aiUKte7RKT7EgrJriOdMCfPJUA6jjwmyhTpOL8bsO7aYWcSwmhz8hLOrqmUahCRmwHMBOAG8JxSaqWIXO97/WkAM+CtvMoBHAVwdbR9Ey1TNKlyw9OuZfhCyzlZrrgDPAAoZE8ekele+dmJaJmbhaHdCsNeG+Tr7dITuJk52jA0wAMAl0sijhpIleudkyW7jqypb0RBtA0yQBreWxJllHQM8ADrrh22fhpRbjISDvIAQCk1A95KKvC5pwN+VgBu0rsvxSfLbcpyh3F7+ooRuP6lRba8NyXH+GM6YPbq3ejdvgCvXz8q43uMh3UvxBM/PA5HahsxoFP8DS5EWpJZRx6qaYgY5HVsnYtdh2r1HgqAd/pCTX16rc/YMjcLh2sb7C4GkWPluF2oa0yv60YyJLsH1JQgj5xjdN92+Kp8H47p3AqThnTG+cd1xTuLtyV83F7tC7Bx7xETSkhmunxkD8xevRuf3ToOAPDUj0Y0reeYSZbdcyaG3jMLUEqzt4woXbTOy8bgLq1xtK4Rz199AuobFQ5W1+HCp+bjkUuGo1tRCxS3ysUFT36NNTur8O2dZyA3y4WWuVlwuwRlmw/g4qfnAwD+dukwjO7bHv/8fAOe/XJj03t0ap2HnYdq8N5No9GtqAXqGxUm/O1zvP7zURjYqRXO+L/PsSHC9fznY3vjn/M2YEzf9njppyei3+9noL5RIT/HjaN13rVefzW+H8o2HUDnNnk4oVdb3P7mMrx948nYf7gOn63djb1VtZi1alfQcf968TAM7tIatQ0eXPHvby36dMlpuBZwsEGdW2PVjkO4alRPvDB/c9BrN5/WF4/PKQcAfH3H6U2Nwv6Goo9uOQUl7QrwxJxyPDNvQ1MQ+Kvx/fD32eswsqQtrj2lF94o24rZq3fjlZ+diI6t81Df6EFVTQOe/3oTpi/bgRE9CrFoSyUmH9sZpw/sgCy3oKFR4VBNPe79IHwJ0fYtc7H3cC2W/uFMtMnPxlfle/FV+V48OXc9fnRiD7z87RYAwN3nDMLEwR1RlJ+D7ZXV6NA6D2t2HEJJ+wLsPFiDKU98BcB7rbn1jaUAgMV3T8Bx930CAFjw+/E44YHZGNu/GM/8+Hgcrm1o+gz+fP6xuKS0W1OnTE19Iwbe/XFTGW8Y1wcfLt2OrQeqw8rfsXUu7j13MD5dvRtvLKzARcd3w2kDOuCmVxbB7RLU792yItL3lfCcPDskMicvU5VMnW7ZsXOyXKhr8P4xlj9wFuas3YPxx3SAiODF+Zvwh/cSH2H71dTTUV3XgNfLKvDMvA2a2/zy9L44d3gXjH9kXsLvpyXwPMlr07TJQY8/W7ML1zyfWX97G/58NlwuwfKKgygqyGaQl2LSfU5esumtH6vrGvF/s9birnMGBT1f1+DBW4sqsHL7Qdx/3rFh+5VMnY4nfjgCk4d2jnr8I7UN2HrgKDq0ykPbghwopfCb15di2oXHYsBdH+P0gR3w3E9OAABs3X8UDR6F7kUt4FHea3Gg/Ufq0LYgJ+i5rfuPYu/hWhzXowil93+CD34xBp3btMDanVW48Kmv2ZNHlIBHLxuOF+dvxu0TB6BD6zzc9sZS/PKMfhjbvxjVdY3IyXKhur4R+w/XoUe7fNTUNyI3y4XdVbVolZeF/JwsbNx7BG1aZKMoPzuuaQsHj9bjcF1DUHIyAPhu43786YOVeOTS4XAJUNwqTzPx4Ny1u3FCSVu8XrYV936wChsfPFvz/f86cy0uKe2OsQ/Pwcp7JyI/xx21nCVTp+OsIZ3wu0kDMe6vc5vuj95bsg2LNh/AvVOGoGTqdHz8q1MwsJN3SseiLQfQu30B2rQI/wx2V9Vgw54jKMzPxsBOrbHvcC2O1jWiqCAHLXOzcPr/zcWAjq3w1BXHN+3z8/+WYdoFQ1FUkIPed0zHA+cfix+e2DNiHckgL0P4g7yND56NXncEj+zJzXKhNoHg5ckfjcCNLy/C1aNL8McfDA567aPlO3DDy4tw+cgeeKNsq+FuaP8fyxNzyvHwzLVhr19/ah/cPnEAXC6xLKB96MKhuP2tZQC4uLxfaJA3f/0+XP6vb9CjbT627D9qU6nM8/7NozXn3FHqYJAXH6vrx8O1DSiIcTMUy6Oz12FUn3YY2autiSXz+n5XFS54kkEekVGXlHbDQxcNs7sYCfN4FA5W16MopIEokFIKHyzbgXNDMvRr2VNVi/wcN/Jz3Jj7/R6cNqCDmcUNU9/ogUsk5tz6aHUkh2tmkLsmHwMRwTlDO+PDZTtw33lD8OOTeja9Pmftblz9nwVxH3ds/2KsuW8S8rLD08Pn+xI+9G5f4O0yNxAY/caX8h2IvN7fsG5t4PL9or/+81GYvXoXnpm3ASNL2uK7Tfvjfk8tjQENHpECvNKeRSjbnO4Z3Y07qXdbfHTLKehS2ALD7p1ld3ES1ldjOQEiiiyepYEiuWV8PxNKos0lgEqhAXYc7pd+rh3TK2h4stNcOarE7iKYwuWSqAEe4M1ZoifAA4DiVs0JE60O8AAg24R8G/Zk7CBL+FtWR/VpBwDoHLJG4MgSY62meVkuzQAPAFr4nne5xPBCy788o7nCz4nwSx20dlavtrjz7GNQ/sBZ+N/PT8KiuycYet+w9xBB7+Loeele+umJpryXXe6bMjj2RvCuRdmxdXgGWBHBMZ1bo3VeZrQPtYjwe01E6cnMbLbxuHxkd83nXTZluD2lX3usuHdi03C2cQOKY+xBfpOGdLK7CLYZ07e9oSzvlJoy406NAKBpgfYfndgTG/YcwQkhQV20RZyjiZa9M8vX85blElMqs0jzobI0evj85Qqdq2HsfVtg4uBO+F/ZVgCRE8DkZbvx6GXDccfby5uSBBjVKjcLVUkeUnR8T32B/qxfj43a+iwimPXrsVhecRC/9U1ATjc/H9vbthtCIrKGADhSm9i1OV5j+xejRbb27ZQdw/7/dukwjOvfAS1zs/CzU3phzto9mHbBUJz04KdJL0s68ngUFt09AW4R/OXjNXjluy12F8ly9583BJ3b5GHcgA5ceieDsCcvQ2yaNhkn92nf9PjucwahTX7whFQrllrIdnmPOaRra8M9eYFCJ9wDwE/H9MLYfta1Qk4c3BFzbh2HNvnZTYFyNN3b5ptzEbToOhpatJd/eiJ+cXpfAEButr7fgVZ52WidF339xf4dW+HC47sZKmMq+MUZ1g0ZIyJ7uGxouHnxmpHIzkqdG+Pzj+vWNEzt5tP74a0bTkanNnko7Vlkc8nCRZihYasGj0Lbghy0yc9GOxMakdPBKf3a44xjOjLAyzAM8igh/gvC8T3bGqpcV947MehxaI/dinsn4q5zBlm6FmDbgpymsc+/Ht88P/DCEdoBTI7bhVTOVxQ4tPa/147E6L7tcdNpfTHn1nFom++MCksPM+YWEVFqsSPIA4DcgDrq8pHdkZPlwme/PTXp5Yg0rx1I1bmBqRNUXDumF2b9eixO9k15AZBKxbNUz3bRp6pQemKQR1G1ijH3qkPAvC2Xgd+mgpAb7dCJpsm4Eb9rcnMK8ayAkxjoWwx7/DEd8fQVI3D5yB4AvL2NZmRuq6qxZqhmfcACpIM6e9P45mW70at9AYoKcjD52Oipz9fcNymu93v4oqHxF5KIyAJ2jcC+Zkwv9GyXjyyX4MELhuL7+8+KOJfdShIlKknFjNGNKdBi+tMxvQAAl5R2R/+OrYKG8TthSP/nt42zuwhkEQZ5FNHFx3fD8nsmRt2mfcvcpjT7ZnTzZxk8xsUJDBsMDDQl4C/izMEdAQAjexVh0pDOePCCYxMqY7IEVuRardrRbjwW3T0h7huTUYGtnkRENrLrnrwwPwdzfjsOq/7U3EiWzKFv900ZjCFdW2NEz8KI24QumZUKVVmLbBd6t7e3F6lTG2+SugGdwqdrpMBHZDn24mUuBnkUUcs4MyiaMUzGaMrYwN6rRASeQ6QLnxlpbf38PW1mCgzS4v0O83Pib3nmGH4iShV2DdcEvJk0A+eVd2gVnqHYbEv/cCZcAgzp2gbv3Dga/702cgbo0I68VOikcrtcKLE5yOvboSXW//lszddS4TMymxX3HZSaGOQ5zEtRKoBQBTnxBQg19YlnNPMHDF9NPR3v3jRa935mpanWcxQzU2KfMyz60Ekj/Fk/B3ZqpRmQRvuejGZgTTeP//A4u4tARBZIdpD34S/GRHxNRHDvuYMtva62yc/Ghgcn47geRch2u6I2Qt50Wl9cN7Y3AG/9YGdA7He4tkH3OmVWyXK5HNNYmZ/jxvRfjsG6B84C4G0koMzljDs6anJcj0Jd2wm0M11Gc+BoffwFCuF/z66FLTC8e6Hu/cwaQqmn0gsd8pKI3Czr5mxEmktwfJQMa0bmH6TiPI9Yzhlq700FEVkj2XHLkK5tor7udoklZcrJcsXVaAt413+79cwBAIALRnRNmaGIoZnAky1aPoFocxzTzeM/PA6L7p4AEUG224UnfzQCrVswAVkmY5DnMKGJTiJxibnDErVccFzXsOeMtnia1Qrnr4z9wxme+OEIXHpCj6Bt4u3hjCZP55IGZrrGN8ncLKnQGkxEBKTe8LoslxgKFFwSfc7cgjvHY0y/9pE3iFIewJslOhWSiogAPdpqr4+bLFlRorwU+IhMcdvEAThnaJeg6RxnH9s5JX4HyDoM4UmTyyVRUzGb4ZFLh4c91yrXWIueWYGG/ziPXjYcADB5aPhwyqKCHLRvmYO9h+sSfr/CFtYsafDG9aNQ3NL6+SAA0KWwRVLeRy9B9FTh78UxDJiI0kuyGp1aZLtRrWOKgttlrC8o1gAJt8H62eUSrPrTRLy1aFtK9FG5RZrW27WtDDEaiWPVKenA7iGxZA/25JGmRo9qmtuVTG3ys7HkDxPi3s/snrxY8+7MupGItUSFUSeUtE3qZPaSdt6W2FS4aYhVGXctSq2glIjM478GjTXQyxWPe84dFHsjWDdc053AQfNzsuAWa8oVL5eI4YDVLN3bRq4TNu87mvYBHsAEaU7FII80eRTw5NxyW9670MCC3Wb35MU6XqwL5o3j+jStvRONHesoWSGdhmym+hIYRGSc/1pkZSPl8O6FuPSEHk3LB0Vj1c11op1f3tkY9l8LXS7Ak+C87rwsF64a1dPw/h1a5UV87a1FFYaPS2Q3BnkU0eRj06d732iQkRMy79BfH8dqJa2MkWTm6tG9NId6hmrwmLP0Q7y0yjbNtw5gpmOLJlHm0ttQl4iJgzvp3jbL5bKkxyzR83NbVK64yyGCrgkO+c/OcqFPh5YmlSiYHfPmreBJgUXnKfky47eX4nJCSeTsin6t8rJw2cjuSSiNOYxWVqGJXvyTkGMdL9ZcDLdLdAUT0VoQjXr6ihExt3n88vAlBBKpAlTI/6nMzMQ5RJRaJAl3NfGsP2pFj9kZAzsknBgty6JhpPE6UtfoXV8wwfPp1Nr8uhRIr1Eq0SQaSFN6YpDnQHefo28ugZUXN7s7U/yTkHMtbKXTUwn37dASn/x6rKnvqydbltY2ibT0pVMroZnrHBJRavEn8bDymlSQo3+YvdvlMn1QZDzLC0VSmJ+dUgGMSqCJsKqmAe1a5uCVn52Icf2LTSwVUGRg+kgqYhZNZ2KQ50BDuxXq2s6qe+G8bBfm3nqaqcc8xrfkgV5ZbsGcW8fhnRu1My0mej1slZelu0z9OrbS1buqV32jsSGgk+IYghQqjWI8Ispg/qzQVgZ5+XGMBshyielT38xIVNKmhb1r05lNKeDkPu3RLUoSFSO6FFrTQ2il0wd2CHpsVYI3Sn0M8igiq1r5hnYrRI925q6Lc9Hx3eLa3i2CXu0L0D3C+jyJtHo9cP6QqL14r/7spLDnrhlt3tp1+4/oW9ohcFjnqf2L0S6BJRcSaYUlIjKLf5h8o4WXpILceHryzK9Hrz458foi2+1CVU2DCaXRFu/wyURjcv/urfIyK3g1AzvxnItBHmlT1iWoaJ0CrUq/n3yMZceOlZRFK9mKmUMpGnVmKps0pDM2TZuMxXdPwDNXHp/Qez5wnjOSthBRavNfS7tbuFRKQW4cPXkWLA/QIo7hopEkOqcvlvl3nB62NtttEwdE3D7RmNwfJLaNc3hlrDnsxlY5tFfoPUC0xd4ps/GbJ00K8bf+DOmqb3hiPENdrBJrmYZELutj+zXPCbh8ZI+w1/v6soDlBCR9MTOg1hvk+RUV5CA3K7GbhrEmz4MgygQi0lZEPhGRdb7/Ncdli8gkEVkrIuUiMjXg+YdFZI2ILBORd0SkMGmFT3Ordxyy7Njx9FJZHUwZlW3x2nQigp+eEtzjOLpv5LULVYJdef7hufEGwN2Koo8qSvVRKif1ahv23GUnBCfN47JBzpWaVx9KCfEGHnoXZ/3TlMFGipNURjvWNk2bjGO7tWl63KVN883AhEEd8cHNY9C5jbeFuUOr5uGRZl6DOT+OKGVMBfCpUqofgE99j4OIiBvAEwDOAjAIwOUi4s+O9QmAIUqpoQC+B3BHUkqdAXq1L7Ds2PEEeal6g52MZFn++f8PXzQUm6ZNxvDuhfhq6ul4/+bwufAJD9f07R/vfUu6D2XM1+hVPuvYzvj7pcNx33lDAKTu7yBZj0EeRRTvnDw9Qw67FbUwtNh5srWIsUj5rWf213WcwAny/7qyNCgADPx8zZz/2MgoL6K2Ban/u0cZZQqAF3w/vwDgPI1tRgIoV0ptUErVAXjNtx+UUrOUUv6JU98AiG/ysYPlxriGJyKeDL1ZLpepa8ucNsCcUROtkzh3rVNAY2fXwhbo2S48ALer1krH4ZiB8iP0XJ53XFf8+CTvAvGduXyCYzHIc6jxx3SM+rqCirt3Sc/2OVnW/coF9owlovyBs2IP59QZlGVHGQsfWPGZmdY/nZYzSBb/1/Xpb061tyDkNB2VUjsAwPd/B41tugLYGvC4wvdcqGsAfKT1JiJynYiUiUjZnj17EixyZkiV5QGy3GJqADP1LHPmk3ewaF05ADguZImH0CkEbVpk492bvL15j1423JT39A+rjPdbL8xP70QtseaHLvj9ePzn6hOSVBpKNQzyHOrOswdG30DFX0lm6Zh7kOjcr2jMqtT1nIfWkJDnflIa9tyPR3lb0kIzai74/Xg8e1Xz9pkwXDNSi2Iq8A9X4Rp5ZDYRmS0iKzT+TdF7CI3ngv6KReT3ABoAvKx1AKXUM0qpUqVUaXEx58cCpq9aYFi2W0yd16WVuCvVLN5a2fTzgxcci5N6twvbpra+EQAwZXhwe4bRoM9ovdclRi9XMurTjq2NN1DnxWg4L26Vm9ReW0ot9mfAIFvoSc4Rb9Ck5yY/x8LJ3nrv382IBb/ZsC/sudMHhveO5vmGDPUpDh6eUhzS66h3PqMerW1a/+ji47vhhfmbbXnvWBo9Cv++sjQlMrtSZlFKjY/0mojsEpHOSqkdItIZwG6NzSoABGZK6AZge8AxrgJwDoAzVKLZKRwkRTryvPWoid+aHTFetltQb3BNCq3kYwAwsFPrsOybAuONtf6/DLO/92T8wSXSQO0OGS30wjUjEy0OZRD25DlUrHlbCvFfeLrqGPdt1bIMgP4hlKeakAlyT1Wt7m03TZscc2iMmUso/DBCpWq1eNKKJ5tHAeMHdTT1cybS4X0AV/l+vgrAexrbLADQT0R6iUgOgMt8+0FEJgH4HYBzlVJHk1BeMpnZw0bzspN/23brmQMw/47TTT1mm/xsPHb5cUHPJfJRGektveWMfnHvY9bw0kCJ9LSFDjwy4/6GMgeDPIeK1RqoVHDSED1+qeOCaeU8Cb2HNqMt3OzzMPNwVgbS0ej5/u1aauGYzq1seV9yvGkAJojIOgATfI8hIl1EZAYA+BKr3AxgJoDVAF5XSq307f84gFYAPhGRJSLydLJPIB31KS4IWsrGTm6XeXPyHjh/CPp1TP61rCA3qykrtJUEggmDoucLiGSUxpDQWDroGCYZ2nluxZpzBbnGpzr4y5OT5TL82VHmSqjpXUTaAvgfgBIAmwBcopQ6oLHdJACPAnAD+LdSyl/R3QPgZwD8M8XvVErNSKRMpE/sNXJU3Gl32+nIXJgKk5zNWB8o9Bh6ejGjSZUkAYnI05HNzuKlmSJ658bwlN1EVlNK7QNwhsbz2wGcHfB4BoCwuk8p1dfSAmaoT387DgDw7JcbscrC9fL0cLnEtHldeupYK+T65n1dNapnzCH52W7B7RNjzPmPRPTVI1r8c+n1ZsssyHXj0tLusTcMYUUjaiKL2/vnmf/slF64zejnThkr0SaJRNcAAoC/KaWG+/4xwEuSWK2BHosSrzxy6fC4jhkPvRWpGclfQrOE/vuq8KQr8XBKPhC7ehmN3jgQUfoKTXhlh0xIquWv77q3jb5wOADUNyr07djS0PuY8lHpPMjgzm103bOEM/9L6NjKeKZT/+9Xui8FQdZINMhLaA0gSm1W3JCnQpanXBPmNGSHVA6J9sQ5Za5YJvRYElGaSIHLjVvMza5ph2M6twYAXDO6F1bcOxHdi6KPXDGaSMxo9XDbxAFJe6/AzKFmOaFXW8P7+u8d8hMY8kmZK9G7XTPWALpZRJaJyHMiUpRgecgkSinbel2sNnFwp4SP8dNTegc91pOtNJoM/ajDWLlOIhFRICuuq6v/NCm+Mpg4XNOOULF/x5bo7xv543IJWuZmxZz7ZbQxz4xGQLO/8tDP/KbTUm8E9fDuhThDI7s3Ucw7LovXAHoKQB8AwwHsAPB/UcrBxV6TyKPsG1pnNTOCvKFd2wQ9TnQBcjN68orys/HhL8YkfJxEfHBz9PfPdrtw/3lDklQaInIyK0ZIxHtIM0cvJFrPGKFV/s37oid6NXrrkA53HFaMRkr0a333ptEY0InJxShczCBPKTVeKTVE4997AHb51v6BkTWAlFK7lFKNSikPgH/BO7QzUjm42GsSKSDuxCt2S+YSUqEVX6JvbcZHXdwqF0NCgs9kO7abve9PRORnRRUWb9DmFjFUN901+Rh8cftpQc/V1id/kTytxt5YwbPRgS1mxMOn9i/GIN/wUiKnS3TsVKJrAHUO2O58ACsSLA+ZRCmVdvOnEhwxGZfWLYIT0w7uklilYsZn3WBwsdpky9QeYiJKLVYko4j3+uVyGR9mGZropLq+0eCRtP1uUuxsjFp1U21D9HIY7XE02vMaGER3aJ2H+883b7SIkVO5a/Ixpr1/LHUNyQ/8KX0kGuQlugbQQyKyXESWATgNwK8TLA+ZJB2Ha57cN/51cowKrYxcCX5WZsTT//3piYkfxET/vlI742hHHWsTERElyop2yriDPDFvTl5/k9fIGzcg9qiokvYFYc/FalBsNHjCeb7M18O6Fxra309Pg6eVbdjXjukV1/aJJOY5WF1neF/KfAkFeUqpfUqpM5RS/Xz/7/c9v10pFbQGkFKqv1Kqj1LqgYDnf6yUOlYpNVQpda4/iQvZTyT9grxHLhludxEMM6MnL9G1+sxy5aieACIvfH7aAG9+phvH9UlamYiI7OA2MfHKyASyMGrp2S72kgh/vXho2HOx1nUzOtXDn/n6uDiDvNDPt8Fjbe9WQYzzFxH0j2MZie5Fsb+HSI7Umtu7S5mFqe4oTI5bDF+kj+tRGPG1C0Z0jfhaOjpjoFYyWWMSjfEeujC8IrbL3ed4l8GM1Ejg7wU1GtgWt2JPIBHFlgoNlS4RQz1byZhinp+TFXMbrXVlH7lkWNR9TuptbFRNrsHsy6EflZ6ePL2fr9ZmL0dZf7FVrvcznfXrU8Nee+pHIzT3idQgqkeNyUN4KbMwyHOwO87SHo9f16hQb3B+14gekVfByLTFOruY2HOWbvMfo/Gfidb9VWCgb/SUE53/SETOkO12obCFvWuzJprAbNO0ySaVxDyF+TnIdkc+r9B1ZPV4+8aT8dxPTjBUntCATc+SRonE0O1b5kR87f+iBMBnHds54mtGxepVJWdjkOdgPz/V/OFy/l4cLSnQqJqyMumzyXK78NYNozQn0Z/Sr7nFsrrOWAtklksSnrNBRM4wqk/y5mprMTpfO9UXUDfaEBzJiB5F6F2sf4hjNA1mZmELiCB/O6F/6FNhju8Zfblno72VWubddhr+EOWei4hBHiVNKnRWXVLazbRjneobYvGL0xNfHDXR9ZxS7Ybg+J7hc0e6tw3u+eyhYz6IFpcINu09omvbVPidIyL7PHXF8bq2+/J3p8XeKImStSLQyBJz5/nZIbT+a9QzJy+Bz7edRk9eac8iLLvnTLRr2TydoCi/uRd58lBvL16sIDAePdrlB70fUSgGeZQ0ndvYnxikIDf2HAS9xg/qCACYNCTxxdUzabhmIP9QpfumDMafzz+26flN0yY3BcnxcrsELXV+jw9dOBRld4039D5E5BzdNJJf2LlWrNEMlfG66uQSU4+3+k+TTD2eHuOP6Rj0uEPrvJj76F3mQWn8rDWXMSfLFbZQ+n+vbc54/efzvPXfi9eMRElAAyezTZOVGOSRJiuqNjN6vBKVxPXS45KZIV5z4oNzh3cNGqoJGJ+jGU88XJCbhfZs6SQiHVJp/ltjyHDIUhN7gAL5e5jMkuw5YmP6tseQrm2CnouWG8CvU5vYgWCoaPcPWvWSf/mJ2b8Ziza+Xr0stwtzb2vuNZ4aITcCkRkY5JGmnATGjV83trfm81kGJmM7RaJZ4FI1ePX3ULbRSH7grxSjTWLXIpCgCvXsIZ3QpVC7ws7UHlIiymyBPXkv//RE/OOHx9lYmsyjd8mlnu3C1wkEgGM6BycA02q09Ffrblfkex//1zyoMxOKkfl4101B/EMHWuUZz0h28fHmzXtLdT8d0wt9TJgs7p+MfUq/9gkfK5VEC179CQmGdG0Td2tyYOyWm+1GUb43UJwyvAvG9G3+DFMhhToRZYYXrxmZtPcKzBA5um/7lJjukIqMtuPpbcjWWicQAOoaghOHaZUjx9ewHW3Yb6o20FJmYJBHQfp3bAUA2Hu41vAx+vmOEahvB3OyZsXywc1jor4+oFN42RJx1zmDkJed+PAUf8rpP/5gcMLHSiXPX30CXv7piZqvuQNqxUR73Py7nzO0S9AwnKwoab6JiADg1jP769puhEVDJrWYmiEyhvwIQyzfuH5UXMeZe+s4E0rT7IUkBtWRaK0TCAA/GNYFpw2IPq9cz+glFfI/kZkY5FGQTjomLOvRuzh4iEOybrUL86P3QF4+skeSShKfbF+rotGep1StIEpL2mJ0X+3eSf8IlnH9i6P+fsT6zgI/MgFQ19CcWS07yjAZIiIAuPn0fk0/P/Pj5mycx3YLnuvlTuLw714RhglaoVuRdi9hXoQAJxL/HDSzJPJpP3rZcLOKoelX4/vjP1c3B6EDNBq3/aIF7PEmbnv+amNrCZIz8Q6IgvxpyhBTjuPPVPbL0/uiKD8bg7iAdVT+hWVb58WX/TOdFwbP8gVgl5zQPeI2a++fFHaO5x/XNWj+w+odVU2PXS6gvrE5yGNPHhFFE7rO2JmDm2+6Lzq+W1AyFqNtRscZWNfzYhOX+4nFn5yqd3EBfhKQbbOuUcdSBBZKJKbWm4E5HtGWKrrj7GMivlYUofH5oQuHxl1OK86LMhd/WyiIWZmxlG+g+YieRbjp9L5JawFN1zwbOW4XCltko7VGgpJoQlM2pxP/70SkLJtnDemE3Cx3WLV66oBiyPTmx6t2HMJQX4u7iAQFedkM8ogoimh1Rui1yWg9ZmQd1ETXTo2Hf2mh60/tg/wcN57/ehOA4AYzOyQyjD/Zc90ijcKJlLHVaCbXZP5eUPpjTx5punxk5N6VeIgIcrPcScusmUhWUDuJCJb88cywcKdQZ9CXjpO33b4ALFKd9VvfPJkG343GQN98ymgVvwCoDRiuWZgfX+ZOInKW3nEkzjI6nD7V8z/5r7V9ils2BbLP/aQ0rmUbFt89wbTyXDumFx684Niwz3tM3/Y4oURfmZJZJeaacN+hdFbijPEoHul5R0yWys9xY9KQxNbO8V+vkn096tAqL61TEYe20r1148kRt/1nwNyRaMNIUlWsVnH/cM4JgzriJyeXNGXQ1Lphavp9E0FNvTfr2dI/nmlK5lMiSm/v3TQ64mun9o+cPCP0EmW0F6Vs8wFD+yXLwE6tsWnaZBzfs6jpHE8f2DFq4+x/rw1OimJmDdS9bT4uH9kDI0vaBj3/1BUjwho0U6Fn67aJA+wuApEmBnkUZtWfJkWt+PTw+K7Edlx/A+cUpLtYw1VSoH4zzN9KK6Kd9dQ/n65bUT7uOXdw07lGq9RdAlT7gjyttfmIyHmGGZgTByS/kTIV6O2tPKVf8D2C3p6oeLhCypLI0k5miHSKbQuSN2IkHUftkH0Y5Dmcf8K12ZqCPBuqyXQOfEKLrncOyOkDO5hfGIs1BXkQ3DiuT9jroUNvI1Vuj142vOlzc4mgX4dWESe6EyWTiLQVkU9EZJ3vf82xZiIySUTWiki5iEzVeP1WEVEiklkLadqg7K7xurdN1tI/qcTozIr8nOSkeNAb45gZdD50kfZaeQDw7Z1n4Pzjupr2XtE8fcUIDDfYYEHOxCDP4a4d08uS4/qvr7GWNLBComuu2Sl8eFD07UtL2qJX+4K0XCg3aOkDjRNtEWX9wcCtC/Nzmnv5ADx80VB8c+cZ5hSSKDFTAXyqlOoH4FPf4yAi4gbwBICzAAwCcLmIDAp4vTuACQC2JKXEGS6e+qE0ZLigE2QZTCFqVtK2eA2MsPatmR1el5R2j3jMjq3zTBkyqieJ2qQhneF2SdS6kSgQs2s63A3j+uCGcX2wcPMB7DpUY9pxT+7THuW7D2NI1zaxNzaZ0cnxqSh0uEqo30zoj1+d0S/qNqlKRNC7uCDi96V3kXmXBKxDJN4FaHlhoxQxBcA4388vAJgL4Hch24wEUK6U2gAAIvKab79Vvtf/BuB2AO9ZXNaMd0JJUVMDUfuWOdh7uE5zu/l3nN40B9hpRvdtj1d+dmJc+3z8q1MsKk10a++flDFroT56+XCMevAzu4tBGSYz/jooYcf3LMLZxyaWbCXQLeP7YaGJ2bbiESswSmWhLYL52W5cPrIHbj6tb9i2/t7SdD7fz347LmKQlx0ybihSY6lLBAeOeG/WGho5YYFSSkel1A4A8P2vNa66K4CtAY8rfM9BRM4FsE0ptTTam4jIdSJSJiJle/bsMafkGejkPu3RIseN9i1zUNqzbcR1STu3aaG7kSnTuF2Ck/voGxV81aieALyJW+yQm+WOWP+l29y1wEaFeLKaEkXDBm/KOKcNSCxpTCopKsjBgxccCwB4fE65zaVJHYHBsAT05NU12LuuEzmPiMwG0Enjpd/rPYTGc0pE8n3HODPWAZRSzwB4BgBKS0vT7PY2eRo8HuRlu1F21wTUN3rQ6EnNj+rkPu3sLoIuPx5Vghfmb07qe+qda3dCSREuPr4b3lhYYdp7D+5ifTDrdgmKkpjIhTIbe/Io47TKy8bVo0vsLgaZLLBuH9ateRhw5zYtmuZK1Nm8eC85j1JqvFJqiMa/9wDsEpHOAOD7f7fGISoABC5M2g3AdgB9APQCsFRENvmeXyQiWgEl6TAgoMcp2+2ypbcuJ8sVc07VtAsiJ/pIJX07tMS3SZ7/rDcsb9cyFw9fPEzztZa58fdvbJo2GRMHW/en52+3nP2bU9OuF5JSF4M8ojTzxA9H2F0ESxzXo1D3tn86b0jTz73aFzT9nI5ZRimjvQ/gKt/PV0F7Xt0CAP1EpJeI5AC4DMD7SqnlSqkOSqkSpVQJvMHgCKXUzmQUPNNsmjYZ5w7rYnj/RJKIjezVnMAlx+3Ca9edFHX7dJpm1rF1nt1FiFunNqlXZn8m8o6tc3H7pOjr7p13XBdMNnF6DWWuNLqUEOmXqS1hb90wCpOHZubFPTBY09KuZfMQlrClJXzft1Pn0VDKmgZggoisgzdD5jQAEJEuIjIDAJRSDQBuBjATwGoAryulVtpUXtIwYVBHzLv9NMP7S8jPWuuCBsqk5GFmefEa7+LrZtTtPz6pZ+IHMZn/K892u9C/Y/TfjwcvGIonfpSZjb1kLs7JI0oTEwd3RL8YF/9MlpsVOYBTpibMJjKHUmofgLDxbEqp7QDODng8A8CMGMcqMbt8pM+/rixNaP/ANikPVMwgTu/6qE6iQv436neTBuKqk0sSPIr5/PPMsxjgk4nYk0cZKdLaOemkVUjmt3/+uLRpLZ12BTkZcY7xCAzkeA9EROmott4TM4gzY921TPHRLd7lGcxa3LzRk5rztt0uwSe/HsvvnkzFII8y0qUndI+9UYp76drIaxUtvHsCSmIMb0w3oplksNm4AR2aAtvQbTN1eC4Rpb/zj+va9HODR8Vc9obDNZsd07k1SnsWYVBnX9Ic38X+mzuMJXypqU/NIA+Ao0fqkDUY5FFGyoTWMKdV9LG+snOHdcHHvxqbnMIQEZlk8tD4Er5wuGawN284GR18CV78q14YTZ5SU99oVrGIUh6DPKIUleV2ZkX/q/H9Ym4TIe8KEVHKibe9Lp2ya6abG8b1sbsIREnDSwlRinLqBGyPjgWKQ7NoOvOTIqJ04IqzZ85pozjiEW+SrdA1Cdu1zDWzOEQpjUEeUYo50bemktthzbn+2xodMR4AoEvAcB325BFRqsrLduNPUwYHPbdp2mSc0q+95vbxBoVOEm/elJ+f2tuagljs6SuOt7sIlAGcdRdJlAb8rbhOm5fR0pdNtFFnFpUzB3eysjhERKYpaReeKKsgJziD8rs3jQbAIC+aeBv0zh0W33zIVNG7OLMSq5E9GOQRpZimIM9hc/JumzgAs39zKjw6g7zA+yCz0msTEVnhGH92yAANId1S/ksah2uap3dxS7uLQGSbhII8EWkrIp+IyDrf/0URtntORHaLyAoj+xM5icuhi6Lm52Shb4eWupdDCFxGgSEeEaWy4lbhc8HC5hYLkJPlijtRi5OwQY9Iv0R78qYC+FQp1Q/Ap77HWp4HMCmB/Ykco6knz6E1vZ7EKwAXRCei9NYqLyvsue/vPysjlgCyilNiPP4GkBkSDfKmAHjB9/MLAM7T2kgpNQ/AfqP7EzlJfo63dbdNi2ybS2KP0f3ao1+H2ENsXEHDNS0sEBGRBXjdil+82TXTVWCc73YJPv3tqfYVhtJWeDNSfDoqpXYAgFJqh4h0sGp/EbkOwHUA0KNHD6PlJUp59583BDeO64tstzOnzJ42oANOGxD7UsLWbiJKZ3rnH1MzJ35kxS1z0YdzC8mAmHeRIjJbRFZo/JuSjAL6KaWeUUqVKqVKi4uLk/nWlKZm/yY9W74K83MwqEv4JH0KFhjicZ4GEaUbXrbi55yPLHDOuXPOmswVsydPKTU+0msisktEOvt64ToD2B3n+ye6P1FEWhPdKYMERHkPXzwMh2sa7CsLEZEOl5R2a/q5bcucoNeEM7FickqDnn+gykvXnqg5d5NIj0R/c94HcBWAab7/30vy/kQRcTRfZgu8IZrINfOIKMV9PfV0dGyd1/T4NxP64ycnl+AH//gK2W5Bj7b5NpYuPTgjxAN6ty/AMz8+HmP6tbe7KJTGEg3ypgF4XUSuBbAFwMUAICJdAPxbKXW27/GrAMYBaC8iFQD+qJR6NtL+RGbggrKZzaHJR4koTXUpbBH0ODfLjc5tWqDsrogDpihUAlHezaf1Na8cFhMRnMnGS0pQQkGeUmofgDM0nt8O4OyAx5fHsz+RGRgDZDbG8EREzmIkxrt8ZA+8+t0Wzm0jx3Fm+j5yBPbkZbaCXM5TICJyEiNz8jq2zvXta3ZpiFIb75IoYzHGy2zXjumF8cd0tLsYRESUJB4DgZp//jZjPHIaBnlElJZys9zo37GV3cUgIqIkOWdoZ6zaccjQvuzJI6dhkEeOMfNXY+0uAhERERl0+6SBce/jH9Uz+djOJpeGKLVxTh5lrGx38K/3gE7s9SEiInKiY7u1sbsIREnFII8ylps59onIRiLSVkQ+EZF1vv+LImw3SUTWiki5iEwNee0XvtdWishDySk5ERGlOwZ55AgXHd/N7iIQkfNMBfCpUqofgE99j4OIiBvAEwDOAjAIwOUiMsj32mkApgAYqpQaDOCvySo4ERGlNwZ55AidWufZXQQicp4pAF7w/fwCgPM0thkJoFwptUEpVQfgNd9+AHADgGlKqVoAUErttra4RESUKRjkUUb79fj+dheBiJyro1JqBwD4/u+gsU1XAFsDHlf4ngOA/gBOEZFvReRzETlB601E5DoRKRORsj179phYfCIiSlfMrkkZ7fie3ikwXDOPiKwgIrMBdNJ46fd6D6HxnD/ZexaAIgAnATgBwOsi0luFrAitlHoGwDMAUFpaykTxRAFY/ZNTMcijjMbcK0RkJaXU+EivicguEemslNohIp0BaA23rADQPeBxNwDbA1572xfUfSciHgDtAbC7jkgnNvKSU3G4JmU08V3dLxvZw+aSEJEDvQ/gKt/PVwF4T2ObBQD6iUgvEckBcJlvPwB4F8DpACAi/QHkANhrZYGJiCgzMMijjNajXT4AoGthC5tLQkQONA3ABBFZB2CC7zFEpIuIzAAApVQDgJsBzASwGsDrSqmVvv2fA9BbRFbAm5DlqtChmkQU3fE926J9yxy7i0GUdJKO9UVpaakqKyuzuxhERGQxEVmolCq1uxzpgvUjEZFzRKsj2ZNHRERERESUQRjkERERERERZRAGeURERERERBmEQR4REREREVEGYZBHRERERESUQRjkERERERERZRAGeURERERERBmEQR4REREREVEGScvF0EWkCsBau8thk/YA9tpdCBs5+fydfO6As8/fyefeUylVbHch0oXD60fA2X8rTj53wNnn7+RzB5x9/hHryKxkl8QkayOt7p7pRKTMqecOOPv8nXzugLPP38nnTnFzbP0IOPtvxcnnDjj7/J187gDPPxIO1yQiIiIiIsogDPKIiIiIiIgySLoGec/YXQAbOfncAWefv5PPHXD2+Tv53Ck+Tv9dcfL5O/ncAWefv5PPHeD5a0rLxCtERERERESkLV178oiIiIiIiEgDgzwiIiIiIqIMklZBnohMEpG1IlIuIlPtLo+VYp2riIwTkYMissT37w92lDNZROQ5EdktIivsLouVYp2nA7/37iIyR0RWi8hKEbnF7jJZRc+5Ou37p/iwjgx63TF/K06pHwHWkaGcUkeyfjQmbebkiYgbwPcAJgCoALAAwOVKqVW2FswCes5VRMYBuFUpdY4dZUw2ERkL4DCAF5VSQ+wuj1VinacDv/fOADorpRaJSCsACwGcl6F/9zHP1WnfP+nHOtK5daRT6keAdWQop9SRrB+NSaeevJEAypVSG5RSdQBeAzDF5jJZxUnnqotSah6A/XaXw2pOOU+9lFI7lFKLfD9XAVgNoKu9pbKGk86VLOGkesNJ5xqTk+oNJ52rHk6pN5xynmZLpyCvK4CtAY8rkLlfsN5zHSUiS0XkIxEZnJyiUQpw5PcuIiUAjgPwrc1FsVyMc3Xk908xsY4Mx78VZ3Lk9+6UOpL1o35ZdhcgDqLxXHqMNY2fnnNdBKCnUuqwiJwN4F0A/awuGNnOkd+7iLQE8BaAXymlDtldHivFOFdHfv+kC+vIYPxbcSZHfu9OqSNZP8YnnXryKgB0D3jcDcB2m8pitZjnqpQ6pJQ67Pt5BoBsEWmfvCKSHZz4vYtINrwX9ZeVUm/bXR4rxTpXJ37/pBvryAD8W3EmJ37vTqkjWT/GL52CvAUA+olILxHJAXAZgPdtLpNVYp6riHQSEfH9PBLe73Jf0ktKSeW07913rs8CWK2UesTu8lhJz7k67funuLCODMC/FWdy2vfulDqS9aMxaTNcUynVICI3A5gJwA3gOaXUSpuLZYlI5yoi1/tefxrARQBuEJEGANUALlPpkirVABF5FcA4AO1FpALAH5VSz9pbKvNpnSeAbMCZ3zuA0QB+DGC5iCzxPXenr5Uu02ieK4AegGO/f9KJdaRz60in1I8A60gNTqkjWT8akDZLKBAREREREVFs6TRck4iIiIiIiGJgkEdERERERJRBGOQRERERERFlEAZ5REREREREGYRBHhERERERUQZhkEdkExFpJyJLfP92isg238+HReRJu8tHRERkB9aPRInjEgpEKUBE7gFwWCn1V7vLQkRElCpYPxIZw548ohQjIuNE5EPfz/eIyAsiMktENonIBSLykIgsF5GPRSTbt93xIvK5iCwUkZki0tnesyAiIjIX60ci/RjkEaW+PgAmA5gC4CUAc5RSxwKoBjDZV5H9A8BFSqnjATwH4AG7CktERJQkrB+JIsiyuwBEFNNHSql6EVkOwA3gY9/zywGUABgAYAiAT0QEvm122FBOIiKiZGL9SBQBgzyi1FcLAEopj4jUq+aJtB54/4YFwEql1Ci7CkhERGQD1o9EEXC4JlH6WwugWERGAYCIZIvIYJvLREREZDfWj+RYDPKI0pxSqg7ARQD+IiJLASwBcLKthSIiIrIZ60dyMi6hQERERERElEHYk0dERERERJRBGOQRERERERFlEAZ5REREREREGYRBHhERERERUQZhkEdERERERJRBGOQRERERERFlEAZ5REREREREGeT/AVDIlkrBgWNLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1080x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# pick a random waveform, but same one from native and augmented set for easier comparison\n",
    "plt.figure(figsize=(15,4))\n",
    "plt.subplot(1, 2, 1)\n",
    "librosa.display.waveplot(waveforms[12], sr=sample_rate)\n",
    "plt.title('Native')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "# augmented waveforms are 2D len 1440 list with 2 waveforms in each position\n",
    "librosa.display.waveplot(augmented_waveforms_temp[0][0], sr=sample_rate)\n",
    "plt.title('AWGN Augmented')\n",
    "plt.show()\n",
    "\n",
    "plt.figure(figsize=(15,4))\n",
    "plt.subplot(1, 2, 1)\n",
    "librosa.display.waveplot(augmented_waveforms_temp[2][0], sr=sample_rate)\n",
    "plt.title('AWGN Augmented')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "librosa.display.waveplot(augmented_waveforms_temp[7][0], sr=sample_rate)\n",
    "plt.title('AWGN Augmented')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z-GOeKiP60DV"
   },
   "source": [
    "Looks noisy alright. Noise is clearly visible in otherwise-silent regions of the waveform. We can see the variability of the noise, which should have an SNR between 15 and 30.\n",
    "\n",
    "**Note that augmentation was only done after splitting data into train, validation, and test sets - and we processed each set separately.**\n",
    "\n",
    "**When we augmented the data before splitting it, test and validation data leaked into the training set giving a 97% test accuracy after training.**\n",
    "\n",
    "## Format Data into Tensor Ready 4D Arrays\n",
    "We don't have a colour channel in our MFCC feature array of dim (#samples, #MFC coefficients, time steps). **We have an analog of a black and white image: instead of 3 colour channels, we have 1 signal intensity channel: magnitude of each of 40 mel frequency coefficients at time t.**\n",
    "\n",
    "**We need an input channel dim to expand to output channels using CNN filters. We create a dummy channel dim to expand features into 2D-CNN-ready 4D tensor format: N x C x H x W.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 50
    },
    "id": "PQCm9rLx60DW",
    "outputId": "8dd1b6a1-16d5-44c5-e579-790aa83dc05a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of 4D feature array for input tensor: (3441, 1, 40, 282) train, (429, 1, 40, 282) validation, (450, 1, 40, 282) test\n",
      "Shape of emotion labels: (3441,) train, (429,) validation, (450,) test\n"
     ]
    }
   ],
   "source": [
    "# need to make dummy input channel for CNN input feature tensor\n",
    "X_train = np.expand_dims(features_train,1)\n",
    "X_valid = np.expand_dims(features_valid, 1)\n",
    "X_test = np.expand_dims(features_test,1)\n",
    "\n",
    "# convert emotion labels from list back to numpy arrays for PyTorch to work with \n",
    "y_train = np.array(y_train)\n",
    "y_valid = np.array(y_valid)\n",
    "y_test = np.array(y_test)\n",
    "\n",
    "# confiorm that we have tensor-ready 4D data array\n",
    "# should print (batch, channel, width, height) == (4320, 1, 128, 282) when multiples==2\n",
    "print(f'Shape of 4D feature array for input tensor: {X_train.shape} train, {X_valid.shape} validation, {X_test.shape} test')\n",
    "print(f'Shape of emotion labels: {y_train.shape} train, {y_valid.shape} validation, {y_test.shape} test')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "id": "3mHDd2jiB9o6"
   },
   "outputs": [],
   "source": [
    "# free up some RAM - no longer need full feature set or any waveforms \n",
    "del features_train, features_valid, features_test, waveforms, augmented_waveforms_temp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Mr8NcUd60Dd"
   },
   "source": [
    "## Feature Scaling\n",
    "Scaling will drastically decrease the length of time the model needs to train to convergence - it will have easier computations to perform on smaller magnitudes. **For reference, scaling reduces the time to convergence from about 500 to 200 epochs for this model.**\n",
    "\n",
    "**Standard Scaling makes the most sense because we have features whose target distribution we don't know.** When I performed classification on this dataset with an MLP classifier standard scaling was best across a variety of conditions and features.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 67
    },
    "id": "tlOPCDUx60De",
    "outputId": "29dfd276-9296-4847-bc73-66d577a8ef4c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train scaled:(3441, 1, 40, 282), y_train:(3441,)\n",
      "X_valid scaled:(429, 1, 40, 282), y_valid:(429,)\n",
      "X_test scaled:(450, 1, 40, 282), y_test:(450,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "\n",
    "#### Scale the training data ####\n",
    "# store shape so we can transform it back \n",
    "N,C,H,W = X_train.shape\n",
    "# Reshape to 1D because StandardScaler operates on a 1D array\n",
    "# tell numpy to infer shape of 1D array with '-1' argument\n",
    "X_train = np.reshape(X_train, (N,-1)) \n",
    "X_train = scaler.fit_transform(X_train)\n",
    "# Transform back to NxCxHxW 4D tensor format\n",
    "X_train = np.reshape(X_train, (N,C,H,W))\n",
    "\n",
    "##### Scale the validation set ####\n",
    "N,C,H,W = X_valid.shape\n",
    "X_valid = np.reshape(X_valid, (N,-1))\n",
    "X_valid = scaler.transform(X_valid)\n",
    "X_valid = np.reshape(X_valid, (N,C,H,W))\n",
    "\n",
    "#### Scale the test set ####\n",
    "N,C,H,W = X_test.shape\n",
    "X_test = np.reshape(X_test, (N,-1))\n",
    "X_test = scaler.transform(X_test)\n",
    "X_test = np.reshape(X_test, (N,C,H,W))\n",
    "\n",
    "# check shape of each set again\n",
    "print(f'X_train scaled:{X_train.shape}, y_train:{y_train.shape}')\n",
    "print(f'X_valid scaled:{X_valid.shape}, y_valid:{y_valid.shape}')\n",
    "print(f'X_test scaled:{X_test.shape}, y_test:{y_test.shape}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zvZh_Cuv7zAb"
   },
   "source": [
    "## Save and Reload Data as NumPy Arrays \n",
    "We can save the training/validation/test data as numpy arrays to enable faster loading in case the notebook kernel crashes / google colab runtime crashes / any number of reasons the training data might be cleared from memory. This is much faster than loading 1440 files and computing their features again - not to mention augmented features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "yx4vK5ssRIzH",
    "outputId": "8a317559-eb79-43e4-c664-d4815d0f33bd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features and labels saved to features+labels.npy\n"
     ]
    }
   ],
   "source": [
    "###### SAVE #########\n",
    "# choose save file name \n",
    "filename = 'features+labels.npy'\n",
    "\n",
    "# open file in write mode and write data\n",
    "with open(filename, 'wb') as f:\n",
    "    np.save(f, X_train)\n",
    "    np.save(f, X_valid)\n",
    "    np.save(f, X_test)\n",
    "    np.save(f, y_train)\n",
    "    np.save(f, y_valid)\n",
    "    np.save(f, y_test)\n",
    "\n",
    "print(f'Features and labels saved to {filename}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 67
    },
    "id": "bj46pQstRKlI",
    "outputId": "02dd2e23-ede4-45aa-87ef-1e62fa195994"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train:(3441, 1, 40, 282), y_train:(3441,)\n",
      "X_valid:(429, 1, 40, 282), y_valid:(429,)\n",
      "X_test:(450, 1, 40, 282), y_test:(450,)\n"
     ]
    }
   ],
   "source": [
    "##### LOAD #########\n",
    "# choose load file name \n",
    "filename = 'features+labels.npy'\n",
    "\n",
    "# open file in read mode and read data \n",
    "with open(filename, 'rb') as f:\n",
    "    X_train = np.load(f)\n",
    "    X_valid = np.load(f)\n",
    "    X_test = np.load(f)\n",
    "    y_train = np.load(f)\n",
    "    y_valid = np.load(f)\n",
    "    y_test = np.load(f)\n",
    "\n",
    "# Check that we've recovered the right data\n",
    "print(f'X_train:{X_train.shape}, y_train:{y_train.shape}')\n",
    "print(f'X_valid:{X_valid.shape}, y_valid:{y_valid.shape}')\n",
    "print(f'X_test:{X_test.shape}, y_test:{y_test.shape}') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aYjV10O460Dg"
   },
   "source": [
    "# Architecture Overview\n",
    "<img src=\"reports/cnn-transformer-final.png\">\n",
    "As a whole, the CNN architecture of this network is inspired by a combination of the golden standards in image and sequence processing over the last few years.\n",
    "\n",
    "Each 3-layer deep 2D convolutional block is extremely similar to the **classic [LeNet](http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf) architecture: Conv->Pool>Conv>Pool>FC.** \n",
    "\n",
    "**[AlexNet](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) forms the basis for increasing the complexity of feature maps with channel expansion through stacked CNN layers; [Inception and GoogLeNet](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43022.pdf) are the inspiration for parallelizing CNN layers in the hopes of diverisfying the features learned by the network.** \n",
    "\n",
    "**[VGGNet](https://arxiv.org/pdf/1409.1556.pdf) proved the unreasonable efficiency of using fixed sized kernels throughout deeply stacked CNN layers;** I found this to extend to this task. Specifically, VGG saw an improvement over AlexNet largely by replacing large kernels (i.e. 11x11 stride 5) with smaller ones of 3x3 stride 1. One of the motivations that VGG cites for this is that the 3x3 kernel is the smallest kernel size choice in understanding spatial data w.r.t. up/down/left/right (although VGG also uses 1x1 kernels). VGGNet also inspires the maxpool kernel size of 2x2 stride 2, as I have used at the first layer of each convolutional block. \n",
    "\n",
    "To be more precise, the motivation to use small stacked filters is two-fold: Computational efficiency and expressivity of feature representation. When we stack 3 3x3 kernels on top of eachother as in this architecture, the second layer has a 5x5 view of the original input volume, and the 3rd layer a 7x7 view. However, the nonlinearities between each smaller layer convey more complex feature representations, whereas a single 7x7 layer would only perform a linear transformation itself. Furthermore, if we keep channel (C) consistent between layers, then 3 3x3 kernels are parameterized by (3(C(3x3xC)) = 27C^2 parameters, while just one 7x7 kernel needs C(7x7xC) = 49C^2 parameters. **Ultimately, small stacked kernels appear to be both more powerful and efficient - although, in [Large Kernel Matters -\n",
    "Improve Semantic Segmentation by Global Convolutional Network](https://arxiv.org/pdf/1703.02719.pdf), the authors conclude that a larger kernel outperforms smaller stacked kernels for semantic segmentation - however, since we are just doing the semantic part (classification) and don't care about \"where\" the emotion is - this shouldn't apply.**\n",
    "\n",
    "Finally, the original [2015 Batch Normalization (BN) paper](https://arxiv.org/abs/1502.03167) suggests that \"We add the BN transform immediately before the nonlinearity\" i.e. before ReLU; however, **I achieved better performance out of this architechture using BN after ReLU. See Keras author's [Francois Chollet's response on GitHub](https://github.com/keras-team/keras/issues/1802#issuecomment-187966878) regarding the BN order issue: \"I can guarantee that recent code written by Christian \\[Szegedy\\] applies relu before BN\".** \n",
    "\n",
    "The Transformer architecture is precisely as in Viswani et al, 2017: Attention is All You Need, but I use 4 stacked encoders instead of 6 as in their paper. For more details on the Transformer block: [Appendix B: The Transformer and Self-Attention (is All You Need)](#Appendix-B:-The-Transformer-and-Self-Attention-(is-All-You-Need))\n",
    "\n",
    "# CNN Motivation\n",
    "**CNNs with 2D convolutional layers are the gold standard for image processing**, other than the recent advances in the Transformer for images. 2D convolution layers accept input feature maps in a (N,C,H,W) (batch size, channel, height, width) format. We have 4320 MFCC plots - 1440 native and 2880 noise augmented - each MFCC plot is of shape 40x282 with 40 MFC coefficients representing different mel pitch ranges, with 282 timesteps for each MFC coefficient. **We can imagine MFCC plots to be a black and white image with 1 signal intensity channel.** Our MFCC input feature tensor will thus be of shape (4320, 1, 40, 282) before splitting for training. I'm going to refer to input/output feature maps and input/output volumes interchangeably, but they have the same meaning. After an activation function operates on a feature map, it produces an activation map.\n",
    "\n",
    "I'm using 3x3 kernels in all 3 layers in both CNN blocks. The first layer only has a single input channel creating a 1x3x3 filter, with 16 output channels necessitating 16 such unique 1x3x3 filters with 1x3x3=9 weights per filter. The next layer has 16 input channels and 32 output channels, producing 32 unique 16x3x3 filters, *each* filter having 16x3x3 = 144 weights. That is, the second layer is applying 32 differently weighted 16x3x3 filters to an input volume of 16x20x141 (the 2x2 maxpooled output of the first layer), producing an output feature map of 32x5x35 after 4x4 stride 4 maxpooling. The last layer has 32 input channels, so a 32x3x3 filter, and 64 output channels, so 64 unique such filters with 32x3x3=288 weights each. The last layer produces an output feature map of 64x1x8 after 4x4 stride 4 maxpooling. **I hope that the simultaneous expansion of filter depth/complexity and reduction of feature map volume will provide the most expressive hierarchical feature representation at the lowest computational cost.** \n",
    "\n",
    "**The key motivations and operations of a 2D Convolutional Neural Net are described in detail in [Appendix A: Convolutional Neural Nets Dissected](#Appendix-A:-Convolutional-Neural-Nets-Dissected) in order to explain the motivation behind the necessity of the CNN for this task and the considerations to take when optimizing the hyperparameters of each layer in a CNN.**\n",
    "\n",
    "# Transformer-Encoder Motivation\n",
    "\n",
    "**I use the Transformer-Encoder layer as introduced in [Attention is All You Need](https://arxiv.org/abs/1706.03762) with the hopes that the network will learn to predict frequency distributions of different emotions according to the global structure of the MFCCs of each emotion.** I could have used LSTM-RNNs to learn the sequence of the spectrogram for each emotion, but the network would only learn to predict frequency changes according to adjacent time steps; in contrast, the multi-head self-attention layers of the transformer enable the network to look at multiple previous time steps when predicting the next. This made sense to me because emotions colour the entire sequence of frequencies, not just at one timestep. \n",
    "\n",
    "**I maxpool the input MFCC map to the transformer block to drastically reduce the number of parameters the network needs to learn.**\n",
    "\n",
    "**The key motivations and operations behind the Transformer architecture are described in detail in [Appendix B: The Transformer](#Appendix-B:-The-Transformer)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8ZaPs9Zm60Dg"
   },
   "source": [
    "# Build Model Architecture and Define Forward Pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "id": "Mk7KLkLQ60Dh"
   },
   "outputs": [],
   "source": [
    "#change nn.sequential to take dict to make more readable \n",
    "\n",
    "class parallel_all_you_want(nn.Module):\n",
    "    # Define all layers present in the network\n",
    "    def __init__(self,num_emotions):\n",
    "        super().__init__() \n",
    "        \n",
    "        ################ TRANSFORMER BLOCK #############################\n",
    "        # maxpool the input feature map/tensor to the transformer \n",
    "        # a rectangular kernel worked better here for the rectangular input spectrogram feature map/tensor\n",
    "        self.transformer_maxpool = nn.MaxPool2d(kernel_size=[1,4], stride=[1,4])\n",
    "        \n",
    "        # define single transformer encoder layer\n",
    "        # self-attention + feedforward network from \"Attention is All You Need\" paper\n",
    "        # 4 multi-head self-attention layers each with 40-->512--->40 feedforward network\n",
    "        transformer_layer = nn.TransformerEncoderLayer(\n",
    "            d_model=40, # input feature (frequency) dim after maxpooling 40*282 -> 40*70 (MFC*time)\n",
    "            nhead=4, # 4 self-attention layers in each multi-head self-attention layer in each encoder block\n",
    "            dim_feedforward=512, # 2 linear layers in each encoder block's feedforward network: dim 40-->512--->40\n",
    "            dropout=0.4, \n",
    "            activation='relu' # ReLU: avoid saturation/tame gradient/reduce compute time\n",
    "        )\n",
    "        \n",
    "        # I'm using 4 instead of the 6 identical stacked encoder layrs used in Attention is All You Need paper\n",
    "        # Complete transformer block contains 4 full transformer encoder layers (each w/ multihead self-attention+feedforward)\n",
    "        self.transformer_encoder = nn.TransformerEncoder(transformer_layer, num_layers=4)\n",
    "        \n",
    "        ############### 1ST PARALLEL 2D CONVOLUTION BLOCK ############\n",
    "        # 3 sequential conv2D layers: (1,40,282) --> (16, 20, 141) -> (32, 5, 35) -> (64, 1, 8)\n",
    "        self.conv2Dblock1 = nn.Sequential(\n",
    "            \n",
    "            # 1st 2D convolution layer\n",
    "            nn.Conv2d(\n",
    "                in_channels=1, # input volume depth == input channel dim == 1\n",
    "                out_channels=16, # expand output feature map volume's depth to 16\n",
    "                kernel_size=3, # typical 3*3 stride 1 kernel\n",
    "                stride=1,\n",
    "                padding=1\n",
    "                      ),\n",
    "            nn.BatchNorm2d(16), # batch normalize the output feature map before activation\n",
    "            nn.ReLU(), # feature map --> activation map\n",
    "            nn.MaxPool2d(kernel_size=2, stride=2), #typical maxpool kernel size\n",
    "            nn.Dropout(p=0.3), #randomly zero 30% of 1st layer's output feature map in training\n",
    "            \n",
    "            # 2nd 2D convolution layer identical to last except output dim, maxpool kernel\n",
    "            nn.Conv2d(\n",
    "                in_channels=16, \n",
    "                out_channels=32, # expand output feature map volume's depth to 32\n",
    "                kernel_size=3,\n",
    "                stride=1,\n",
    "                padding=1\n",
    "                      ),\n",
    "            nn.BatchNorm2d(32),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=4, stride=4), # increase maxpool kernel for subsequent filters\n",
    "            nn.Dropout(p=0.3), \n",
    "            \n",
    "            # 3rd 2D convolution layer identical to last except output dim\n",
    "            nn.Conv2d(\n",
    "                in_channels=32,\n",
    "                out_channels=64, # expand output feature map volume's depth to 64\n",
    "                kernel_size=3,\n",
    "                stride=1,\n",
    "                padding=1\n",
    "                      ),\n",
    "            nn.BatchNorm2d(64),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=4, stride=4),\n",
    "            nn.Dropout(p=0.3),\n",
    "        )\n",
    "        ############### 2ND PARALLEL 2D CONVOLUTION BLOCK ############\n",
    "        # 3 sequential conv2D layers: (1,40,282) --> (16, 20, 141) -> (32, 5, 35) -> (64, 1, 8)\n",
    "        self.conv2Dblock2 = nn.Sequential(\n",
    "            \n",
    "            # 1st 2D convolution layer\n",
    "            nn.Conv2d(\n",
    "                in_channels=1, # input volume depth == input channel dim == 1\n",
    "                out_channels=16, # expand output feature map volume's depth to 16\n",
    "                kernel_size=3, # typical 3*3 stride 1 kernel\n",
    "                stride=1,\n",
    "                padding=1\n",
    "                      ),\n",
    "            nn.BatchNorm2d(16), # batch normalize the output feature map before activation\n",
    "            nn.ReLU(), # feature map --> activation map\n",
    "            nn.MaxPool2d(kernel_size=2, stride=2), #typical maxpool kernel size\n",
    "            nn.Dropout(p=0.3), #randomly zero 30% of 1st layer's output feature map in training\n",
    "            \n",
    "            # 2nd 2D convolution layer identical to last except output dim, maxpool kernel\n",
    "            nn.Conv2d(\n",
    "                in_channels=16, \n",
    "                out_channels=32, # expand output feature map volume's depth to 32\n",
    "                kernel_size=3,\n",
    "                stride=1,\n",
    "                padding=1\n",
    "                      ),\n",
    "            nn.BatchNorm2d(32),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=4, stride=4), # increase maxpool kernel for subsequent filters\n",
    "            nn.Dropout(p=0.3), \n",
    "            \n",
    "            # 3rd 2D convolution layer identical to last except output dim\n",
    "            nn.Conv2d(\n",
    "                in_channels=32,\n",
    "                out_channels=64, # expand output feature map volume's depth to 64\n",
    "                kernel_size=3,\n",
    "                stride=1,\n",
    "                padding=1\n",
    "                      ),\n",
    "            nn.BatchNorm2d(64),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=4, stride=4),\n",
    "            nn.Dropout(p=0.3),\n",
    "        )\n",
    "\n",
    "        ################# FINAL LINEAR BLOCK ####################\n",
    "        # Linear softmax layer to take final concatenated embedding tensor \n",
    "        #    from parallel 2D convolutional and transformer blocks, output 8 logits \n",
    "        # Each full convolution block outputs (64*1*8) embedding flattened to dim 512 1D array \n",
    "        # Full transformer block outputs 40*70 feature map, which we time-avg to dim 40 1D array\n",
    "        # 512*2+40 == 1064 input features --> 8 output emotions \n",
    "        self.fc1_linear = nn.Linear(512*2+40,num_emotions) \n",
    "        \n",
    "        ### Softmax layer for the 8 output logits from final FC linear layer \n",
    "        self.softmax_out = nn.Softmax(dim=1) # dim==1 is the freq embedding\n",
    "        \n",
    "    # define one complete parallel fwd pass of input feature tensor thru 2*conv+1*transformer blocks\n",
    "    def forward(self,x):\n",
    "        \n",
    "        ############ 1st parallel Conv2D block: 4 Convolutional layers ############################\n",
    "        # create final feature embedding from 1st convolutional layer \n",
    "        # input features pased through 4 sequential 2D convolutional layers\n",
    "        conv2d_embedding1 = self.conv2Dblock1(x) # x == N/batch * channel * freq * time\n",
    "        \n",
    "        # flatten final 64*1*8 feature map from convolutional layers to length 512 1D array \n",
    "        # skip the 1st (N/batch) dimension when flattening\n",
    "        conv2d_embedding1 = torch.flatten(conv2d_embedding1, start_dim=1) \n",
    "        \n",
    "        ############ 2nd parallel Conv2D block: 4 Convolutional layers #############################\n",
    "        # create final feature embedding from 2nd convolutional layer \n",
    "        # input features pased through 4 sequential 2D convolutional layers\n",
    "        conv2d_embedding2 = self.conv2Dblock2(x) # x == N/batch * channel * freq * time\n",
    "        \n",
    "        # flatten final 64*1*8 feature map from convolutional layers to length 512 1D array \n",
    "        # skip the 1st (N/batch) dimension when flattening\n",
    "        conv2d_embedding2 = torch.flatten(conv2d_embedding2, start_dim=1) \n",
    "        \n",
    "         \n",
    "        ########## 4-encoder-layer Transformer block w/ 40-->512-->40 feedfwd network ##############\n",
    "        # maxpool input feature map: 1*40*282 w/ 1*4 kernel --> 1*40*70\n",
    "        x_maxpool = self.transformer_maxpool(x)\n",
    "\n",
    "        # remove channel dim: 1*40*70 --> 40*70\n",
    "        x_maxpool_reduced = torch.squeeze(x_maxpool,1)\n",
    "        \n",
    "        # convert maxpooled feature map format: batch * freq * time ---> time * batch * freq format\n",
    "        # because transformer encoder layer requires tensor in format: time * batch * embedding (freq)\n",
    "        x = x_maxpool_reduced.permute(2,0,1) \n",
    "        \n",
    "        # finally, pass reduced input feature map x into transformer encoder layers\n",
    "        transformer_output = self.transformer_encoder(x)\n",
    "        \n",
    "        # create final feature emedding from transformer layer by taking mean in the time dimension (now the 0th dim)\n",
    "        # transformer outputs 2x40 (MFCC embedding*time) feature map, take mean of columns i.e. take time average\n",
    "        transformer_embedding = torch.mean(transformer_output, dim=0) # dim 40x70 --> 40\n",
    "        \n",
    "        ############# concatenate freq embeddings from convolutional and transformer blocks ######\n",
    "        # concatenate embedding tensors output by parallel 2*conv and 1*transformer blocks\n",
    "        complete_embedding = torch.cat([conv2d_embedding1, conv2d_embedding2,transformer_embedding], dim=1)  \n",
    "\n",
    "        ######### final FC linear layer, need logits for loss #########################\n",
    "        output_logits = self.fc1_linear(complete_embedding)  \n",
    "        \n",
    "        ######### Final Softmax layer: use logits from FC linear, get softmax for prediction ######\n",
    "        output_softmax = self.softmax_out(output_logits)\n",
    "        \n",
    "        # need output logits to compute cross entropy loss, need softmax probabilities to predict class\n",
    "        return output_logits, output_softmax                       "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-1fIIe6i60Dj"
   },
   "source": [
    "## Analyzing The Flow of Tensors Through the Network\n",
    "We zero-pad 1 the input feature map to each convolutional layer to get back from the layer the same shape tensor as we input: zero-pad 1 adds 2 to each of (H, W) dims, and the 3x3, stride 1 kernels cuts off (kernel - stride == 2) dims from each of (H,W). **Zero-pad 1 --> 3x3 stride 1 kernel effectively throws away the zero pads to get same input/output shape from each conv2D block.**\n",
    "\n",
    "At the end of first convolutional layer in each block we have a maxpool kernel of size 2x2, stride 2 which will take 1 of 4 pixels in its winddow. For the first input feature map the maxpool kernel will progress 40/2 = 20 times over the rows and 282/2=141 times over the columns, producing a 20x141 output map. **Nonoverlapping maxpool kernels reduce each output dim to _input dim / kernel size_.** We then expand the output channels to 16 making an output feature map of (16x20x141). \n",
    "\n",
    "The next two convolutional layers in each block have a maxpool kernel size 4x4, stride 4. Same math as above, maxpool reduces each dim/4. 2nd conv layer takes (16x20x141) --> (32x5x35). 3rd and final conv layer takes (32x5x35) --> (64x1x8).\n",
    "\n",
    "**Note that in (N,C,H,W) format, for MFCCs H = MFCC (pitch),  W = time step.**\n",
    "\n",
    "**Complete flow through each convolutional block (C,H,W):**\n",
    "\n",
    "    Layer 1 ---> 1x40x282 --> PAD-1 --> 1x42x284 --> FILTER 1x3x3 --> 16x40x282 --> MAXPOOL 2x2 stride 2 --> 16x20x141\n",
    "\n",
    "    Layer 2 ---> 16x20x141 --> PAD-1 --> 16x22x143 --> FILTER 16x3x3 --> 32x20x141 --> MAXPOOL 4x4 stride 4 --> 32x5x35\n",
    "\n",
    "    Layer 3 ---> 32x5x35 --> PAD-1 --> 32x7x37 --> FILTER 32x3x3 --> 64x5x35 --> MAXPOOL 4x4 stride 4 --> 64x1x8 \n",
    "\n",
    "    Flatten ---> 64x1x8 --> Final convolutional embedding length 512 1D array\n",
    "\n",
    "\n",
    "**Complete flow through transformer encoder block (C,H,W):**\n",
    "\n",
    "    Maxpool 1x4 stride 1x4 ---> 1x40x282 --> 1x20x70\n",
    "\n",
    "    Drop channel ---> 1x20x70 --> 20x70 (H,W)\n",
    "\n",
    "    Change dims ---> 20x70 --> 70x40 (W,H)\n",
    "\n",
    "    4xTransformer encoder ---> 70x40 --> 2x40 (W,H)\n",
    "\n",
    "    Time average ---> 2x40 --> 1x40 --> Final transformer embedding length 40 1D array\n",
    "\n",
    "**FC Linear network (C,H,W):**\n",
    "\n",
    "    Concatenate ---> 512+512+40 --> 1064 \n",
    "\n",
    "    FC Linear layer ---> 1064 --> Final linear logits output length 8 1D array\n",
    "\n",
    "    Softmax layer: 8 ----> 1 predicted emotion / max probability class\n",
    "\n",
    "\n",
    "We can confirm our network's tensor shapes and flow using the excellent torchsummary package which provides a PyTorch implementation of Keras' model.summary method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "LEu0842Q60Dk",
    "outputId": "bd58f880-5b37-4bed-9839-6a0381070d36"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------\n",
      "        Layer (type)               Output Shape         Param #\n",
      "================================================================\n",
      "            Conv2d-1          [-1, 16, 40, 282]             160\n",
      "       BatchNorm2d-2          [-1, 16, 40, 282]              32\n",
      "              ReLU-3          [-1, 16, 40, 282]               0\n",
      "         MaxPool2d-4          [-1, 16, 20, 141]               0\n",
      "           Dropout-5          [-1, 16, 20, 141]               0\n",
      "            Conv2d-6          [-1, 32, 20, 141]           4,640\n",
      "       BatchNorm2d-7          [-1, 32, 20, 141]              64\n",
      "              ReLU-8          [-1, 32, 20, 141]               0\n",
      "         MaxPool2d-9            [-1, 32, 5, 35]               0\n",
      "          Dropout-10            [-1, 32, 5, 35]               0\n",
      "           Conv2d-11            [-1, 64, 5, 35]          18,496\n",
      "      BatchNorm2d-12            [-1, 64, 5, 35]             128\n",
      "             ReLU-13            [-1, 64, 5, 35]               0\n",
      "        MaxPool2d-14             [-1, 64, 1, 8]               0\n",
      "          Dropout-15             [-1, 64, 1, 8]               0\n",
      "           Conv2d-16          [-1, 16, 40, 282]             160\n",
      "      BatchNorm2d-17          [-1, 16, 40, 282]              32\n",
      "             ReLU-18          [-1, 16, 40, 282]               0\n",
      "        MaxPool2d-19          [-1, 16, 20, 141]               0\n",
      "          Dropout-20          [-1, 16, 20, 141]               0\n",
      "           Conv2d-21          [-1, 32, 20, 141]           4,640\n",
      "      BatchNorm2d-22          [-1, 32, 20, 141]              64\n",
      "             ReLU-23          [-1, 32, 20, 141]               0\n",
      "        MaxPool2d-24            [-1, 32, 5, 35]               0\n",
      "          Dropout-25            [-1, 32, 5, 35]               0\n",
      "           Conv2d-26            [-1, 64, 5, 35]          18,496\n",
      "      BatchNorm2d-27            [-1, 64, 5, 35]             128\n",
      "             ReLU-28            [-1, 64, 5, 35]               0\n",
      "        MaxPool2d-29             [-1, 64, 1, 8]               0\n",
      "          Dropout-30             [-1, 64, 1, 8]               0\n",
      "        MaxPool2d-31            [-1, 1, 40, 70]               0\n",
      "MultiheadAttention-32  [[-1, 2, 40], [-1, 70, 70]]               0\n",
      "          Dropout-33                [-1, 2, 40]               0\n",
      "        LayerNorm-34                [-1, 2, 40]              80\n",
      "           Linear-35               [-1, 2, 512]          20,992\n",
      "          Dropout-36               [-1, 2, 512]               0\n",
      "           Linear-37                [-1, 2, 40]          20,520\n",
      "          Dropout-38                [-1, 2, 40]               0\n",
      "        LayerNorm-39                [-1, 2, 40]              80\n",
      "TransformerEncoderLayer-40                [-1, 2, 40]               0\n",
      "MultiheadAttention-41  [[-1, 2, 40], [-1, 70, 70]]               0\n",
      "          Dropout-42                [-1, 2, 40]               0\n",
      "        LayerNorm-43                [-1, 2, 40]              80\n",
      "           Linear-44               [-1, 2, 512]          20,992\n",
      "          Dropout-45               [-1, 2, 512]               0\n",
      "           Linear-46                [-1, 2, 40]          20,520\n",
      "          Dropout-47                [-1, 2, 40]               0\n",
      "        LayerNorm-48                [-1, 2, 40]              80\n",
      "TransformerEncoderLayer-49                [-1, 2, 40]               0\n",
      "MultiheadAttention-50  [[-1, 2, 40], [-1, 70, 70]]               0\n",
      "          Dropout-51                [-1, 2, 40]               0\n",
      "        LayerNorm-52                [-1, 2, 40]              80\n",
      "           Linear-53               [-1, 2, 512]          20,992\n",
      "          Dropout-54               [-1, 2, 512]               0\n",
      "           Linear-55                [-1, 2, 40]          20,520\n",
      "          Dropout-56                [-1, 2, 40]               0\n",
      "        LayerNorm-57                [-1, 2, 40]              80\n",
      "TransformerEncoderLayer-58                [-1, 2, 40]               0\n",
      "MultiheadAttention-59  [[-1, 2, 40], [-1, 70, 70]]               0\n",
      "          Dropout-60                [-1, 2, 40]               0\n",
      "        LayerNorm-61                [-1, 2, 40]              80\n",
      "           Linear-62               [-1, 2, 512]          20,992\n",
      "          Dropout-63               [-1, 2, 512]               0\n",
      "           Linear-64                [-1, 2, 40]          20,520\n",
      "          Dropout-65                [-1, 2, 40]               0\n",
      "        LayerNorm-66                [-1, 2, 40]              80\n",
      "TransformerEncoderLayer-67                [-1, 2, 40]               0\n",
      "TransformerEncoder-68                [-1, 2, 40]               0\n",
      "           Linear-69                    [-1, 8]           8,520\n",
      "          Softmax-70                    [-1, 8]               0\n",
      "================================================================\n",
      "Total params: 222,248\n",
      "Trainable params: 222,248\n",
      "Non-trainable params: 0\n",
      "----------------------------------------------------------------\n",
      "Input size (MB): 0.04\n",
      "Forward/backward pass size (MB): 2.61\n",
      "Params size (MB): 0.85\n",
      "Estimated Total Size (MB): 3.50\n",
      "----------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "from torchsummary import summary\n",
    "\n",
    "# need device to instantiate model\n",
    "device = 'cuda'\n",
    "\n",
    "# instantiate model for 8 emotions and move to GPU \n",
    "model = parallel_all_you_want(len(emotions_dict)).to(device)\n",
    "\n",
    "# include input feature map dims in call to summary()\n",
    "summary(model, input_size=(1,40,282))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GVZg7zuB60Dm"
   },
   "source": [
    "## Define Loss/Criterion\n",
    "\n",
    "We must define the loss function (criterion per PyTorch notation) for the backwards pass of each training iteration. Since our classes our balanced we don't need to specify a class-weight parameter (to balance classes).\n",
    "\n",
    "**PyTorch nn.CrossEntropyLoss() implements log softmax and negative log likelihood loss (nn.NLLoss() --> nn.LogSoftmax())\n",
    "We use log softmax for computation benefits and faster gradient optimization. Log softmax heavily penalizes the model when failing to predict the correct class.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "id": "F4IudptP60Dm"
   },
   "outputs": [],
   "source": [
    "# define loss function; CrossEntropyLoss() fairly standard for multiclass problems \n",
    "def criterion(predictions, targets): \n",
    "    return nn.CrossEntropyLoss()(input=predictions, target=targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_hF-f9Sl60Do"
   },
   "source": [
    "## Choose Optimizer\n",
    "\n",
    "I used Adam to train an MLP due to its faster compute and convergence. Adam is great and usually works well with defaults.\n",
    "\n",
    "**However, a lot of 2018-2020 papers still use SGD. It seems to me that the reason is SGD with properly tuned momentum sometimes (often) converges to lower loss with enough training.**\n",
    "\n",
    "**Quoting [Wilson et al, 2017](https://arxiv.org/abs/1705.08292):** \n",
    "\n",
    "    “We observe that the solutions found by adaptive methods generalize worse (often significantly worse) than SGD, even \n",
    "    when these solutions have better training performance. These results suggest that practitioners should reconsider the \n",
    "    use of adaptive methods to train neural networks.\"\n",
    "\n",
    "\n",
    "\"Adaptive methods\" refers to the likes of Adam.\n",
    "   \n",
    "**I took full advantage of plain old SGD by using the highest momentum leading to convergence, plus a generously long training time.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "id": "LGFjDT1i60Do"
   },
   "outputs": [],
   "source": [
    "optimizer = torch.optim.SGD(model.parameters(),lr=0.01, weight_decay=1e-3, momentum=0.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mr6Vi4Gs60Dr"
   },
   "source": [
    "## Define Training Step\n",
    "\n",
    "We define a function to return a single training step defining one iteration of our model.\n",
    "\n",
    "    Forward pass output logits and softmax probabilities. \n",
    "\n",
    "    Record the softmax probabilities to track accuracy. \n",
    "\n",
    "    Pass output logits to loss function to compute loss.\n",
    "    \n",
    "    Call backwards pass with loss function (backpropogate errors).\n",
    "    \n",
    "    Tell optimizer to apply one update step to network parameters.\n",
    "    \n",
    "    Zero the accumulated gradient in the optimizer for next iteration.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "id": "zPTbQ53_60Dr"
   },
   "outputs": [],
   "source": [
    "# define function to create a single step of the training phase\n",
    "def make_train_step(model, criterion, optimizer):\n",
    "    \n",
    "    # define the training step of the training phase\n",
    "    def train_step(X,Y):\n",
    "        \n",
    "        # forward pass\n",
    "        output_logits, output_softmax = model(X)\n",
    "        predictions = torch.argmax(output_softmax,dim=1)\n",
    "        accuracy = torch.sum(Y==predictions)/float(len(Y))\n",
    "        \n",
    "        # compute loss on logits because nn.CrossEntropyLoss implements log softmax\n",
    "        loss = criterion(output_logits, Y) \n",
    "        \n",
    "        # compute gradients for the optimizer to use \n",
    "        loss.backward()\n",
    "        \n",
    "        # update network parameters based on gradient stored (by calling loss.backward())\n",
    "        optimizer.step()\n",
    "        \n",
    "        # zero out gradients for next pass\n",
    "        # pytorch accumulates gradients from backwards passes (convenient for RNNs)\n",
    "        optimizer.zero_grad() \n",
    "        \n",
    "        return loss.item(), accuracy*100\n",
    "    return train_step"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xInmb5vS60Dt"
   },
   "source": [
    "## Define Validation Step\n",
    "\n",
    "Define a function to return a single validation step on the 10% X,y tensor pair to get an idea of our model's generalizibiliy as it trains so we know whether and when to stop it and tune hyperparameters. **Make sure we _do not_ update network parameters during validation by setting model to validation mode. Do not waste resources computing gradients in validation phase by setting torch.no_grad().**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "id": "vcQyDxEY60Du"
   },
   "outputs": [],
   "source": [
    "def make_validate_fnc(model,criterion):\n",
    "    def validate(X,Y):\n",
    "        \n",
    "        # don't want to update any network parameters on validation passes: don't need gradient\n",
    "        # wrap in torch.no_grad to save memory and compute in validation phase: \n",
    "        with torch.no_grad(): \n",
    "            \n",
    "            # set model to validation phase i.e. turn off dropout and batchnorm layers \n",
    "            model.eval()\n",
    "      \n",
    "            # get the model's predictions on the validation set\n",
    "            output_logits, output_softmax = model(X)\n",
    "            predictions = torch.argmax(output_softmax,dim=1)\n",
    "\n",
    "            # calculate the mean accuracy over the entire validation set\n",
    "            accuracy = torch.sum(Y==predictions)/float(len(Y))\n",
    "            \n",
    "            # compute error from logits (nn.crossentropy implements softmax)\n",
    "            loss = criterion(output_logits,Y)\n",
    "            \n",
    "        return loss.item(), accuracy*100, predictions\n",
    "    return validate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gQB8sZnL60Dx"
   },
   "source": [
    "# Make Checkpoint Functions\n",
    "It's a good idea to save checkpoints of the model state after each epoch. We'll then interrupt training when satisfied with the model's performance and load the appropriate model binary. \n",
    "\n",
    "- Resume training if hardware/software fails\n",
    "- Save compute re-training by training from checkpoint after tuning\n",
    "- Implement early stopping easily by keeping snapshot of most performant version of model\n",
    "- Google Colab throttles GPU usage eventually; can't keep re-training from scratch indefinitely"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "id": "rNxhD9-p60Dy"
   },
   "outputs": [],
   "source": [
    "def make_save_checkpoint(): \n",
    "    def save_checkpoint(optimizer, model, epoch, filename):\n",
    "        checkpoint_dict = {\n",
    "            'optimizer': optimizer.state_dict(),\n",
    "            'model': model.state_dict(),\n",
    "            'epoch': epoch\n",
    "        }\n",
    "        torch.save(checkpoint_dict, filename)\n",
    "    return save_checkpoint\n",
    "\n",
    "def load_checkpoint(optimizer, model, filename):\n",
    "    checkpoint_dict = torch.load(filename)\n",
    "    epoch = checkpoint_dict['epoch']\n",
    "    model.load_state_dict(checkpoint_dict['model'])\n",
    "    if optimizer is not None:\n",
    "        optimizer.load_state_dict(checkpoint_dict['optimizer'])\n",
    "    return epoch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jOiUNC0k60D3"
   },
   "source": [
    "# Build Training Loop\n",
    "\n",
    "Build the complete training loop using the training and validation step functions. \n",
    "\n",
    "This model is not reasonable to train on CPU, but it's a good way to check if the model compiled successfully. I'm using Google Colab's free GPU (K80 - 24GB RAM ~2.9 TFLOPs). This model is pretty big (383,688 params to learn if I've checked the math correctly) but trains to convergence within ~10 minutes on a K80.\n",
    "\n",
    "Pick the number of epochs (complete pass of all training samples) to use that is higher than reasonable so the model does not terminate just before convergence - I manually stopped it when it converges. \n",
    "\n",
    "**Minibatch size:** [From Yann LeCun's twitter](https://twitter.com/ylecun/status/989610208497360896?lang=en) (with LeCun facebook comment appended) [citing this 2018 minibatch paper](https://arxiv.org/abs/1804.07612):\n",
    "\n",
    "\n",
    "    \n",
    "    \"Training with large minibatches is bad for your health. More importantly, it's bad for your test error. Friends dont \n",
    "    let friends use minibatches larger than 32. Let's face it: the only people have switched to minibatch sizes larger than one since 2012 is because GPUs are inefficient for batch sizes smaller than 32. That's a terrible reason. It just means our hardware sucks.\" \n",
    "    \n",
    "\n",
    "\n",
    "<br>\n",
    "That's about it for the logic. Here's the full training loop:\n",
    "\n",
    "    --Setup--\n",
    "    \n",
    "    Instantiate model.\n",
    "\n",
    "    Instantiate training and validation steps with model, loss function, and optimizer. \n",
    "    \n",
    "    Move model to GPU.\n",
    "    \n",
    "        --Epoch--\n",
    "    \n",
    "        Set model to train mode after each post-epoch validation phase completes.\n",
    "    \n",
    "        Shuffle the training set for each  epoch, reset epoch loss and accuracy. \n",
    "        \n",
    "            --Iteration--\n",
    "    \n",
    "            Create X_train, y_train minibatch tensors for each iteration and move tensors to GPU.\n",
    "            \n",
    "            Take 1 train step with X_train, y_train minibatch tensors.\n",
    "\n",
    "            Aggregate accuracy and loss from each iteration, but only record after each epoch. \n",
    "            \n",
    "        --Epoch--\n",
    "\n",
    "        Compute and record validation accuracy for the entire epoch to keep track of learning progress. \n",
    "\n",
    "        Print training metrics after each epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 50
    },
    "id": "eg7x2boi60D3",
    "outputId": "995820c9-4479-405d-91b4-0aed608dcd86"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda selected\n",
      "Number of trainable params:  248488\n"
     ]
    }
   ],
   "source": [
    "# get training set size to calculate # iterations and minibatch indices\n",
    "train_size = X_train.shape[0]\n",
    "\n",
    "# pick minibatch size (of 32... always)\n",
    "minibatch = 32\n",
    "\n",
    "# set device to GPU\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "print(f'{device} selected')\n",
    "\n",
    "# instantiate model and move to GPU for training\n",
    "model = parallel_all_you_want(num_emotions=len(emotions_dict)).to(device) \n",
    "print('Number of trainable params: ',sum(p.numel() for p in model.parameters()) )\n",
    "\n",
    "# encountered bugs in google colab only, unless I explicitly defined optimizer in this cell...\n",
    "optimizer = torch.optim.SGD(model.parameters(),lr=0.01, weight_decay=1e-3, momentum=0.8)\n",
    "\n",
    "# instantiate the checkpoint save function\n",
    "save_checkpoint = make_save_checkpoint()\n",
    "\n",
    "# instantiate the training step function \n",
    "train_step = make_train_step(model, criterion, optimizer=optimizer)\n",
    "\n",
    "# instantiate the validation loop function\n",
    "validate = make_validate_fnc(model,criterion)\n",
    "\n",
    "# instantiate lists to hold scalar performance metrics to plot later\n",
    "train_losses=[]\n",
    "valid_losses = []\n",
    "\n",
    "# create training loop for one complete epoch (entire training set)\n",
    "def train(optimizer, model, num_epochs, X_train, Y_train, X_valid, Y_valid):\n",
    "\n",
    "    for epoch in range(num_epochs):\n",
    "        \n",
    "        # set model to train phase\n",
    "        model.train()         \n",
    "        \n",
    "        # shuffle entire training set in each epoch to randomize minibatch order\n",
    "        train_indices = np.random.permutation(train_size) \n",
    "        \n",
    "        # shuffle the training set for each epoch:\n",
    "        X_train = X_train[train_indices,:,:,:] \n",
    "        Y_train = Y_train[train_indices]\n",
    "\n",
    "        # instantiate scalar values to keep track of progress after each epoch so we can stop training when appropriate \n",
    "        epoch_acc = 0 \n",
    "        epoch_loss = 0\n",
    "        num_iterations = int(train_size / minibatch)\n",
    "        \n",
    "        # create a loop for each minibatch of 32 samples:\n",
    "        for i in range(num_iterations):\n",
    "            \n",
    "            # we have to track and update minibatch position for the current minibatch\n",
    "            # if we take a random batch position from a set, we almost certainly will skip some of the data in that set\n",
    "            # track minibatch position based on iteration number:\n",
    "            batch_start = i * minibatch \n",
    "            # ensure we don't go out of the bounds of our training set:\n",
    "            batch_end = min(batch_start + minibatch, train_size) \n",
    "            # ensure we don't have an index error\n",
    "            actual_batch_size = batch_end-batch_start \n",
    "            \n",
    "            # get training minibatch with all channnels and 2D feature dims\n",
    "            X = X_train[batch_start:batch_end,:,:,:] \n",
    "            # get training minibatch labels \n",
    "            Y = Y_train[batch_start:batch_end] \n",
    "\n",
    "            # instantiate training tensors\n",
    "            X_tensor = torch.tensor(X, device=device).float() \n",
    "            Y_tensor = torch.tensor(Y, dtype=torch.long,device=device)\n",
    "            \n",
    "            # Pass input tensors thru 1 training step (fwd+backwards pass)\n",
    "            loss, acc = train_step(X_tensor,Y_tensor) \n",
    "            \n",
    "            # aggregate batch accuracy to measure progress of entire epoch\n",
    "            epoch_acc += acc * actual_batch_size / train_size\n",
    "            epoch_loss += loss * actual_batch_size / train_size\n",
    "            \n",
    "            # keep track of the iteration to see if the model's too slow\n",
    "            print('\\r'+f'Epoch {epoch}: iteration {i}/{num_iterations}',end='')\n",
    "        \n",
    "        # create tensors from validation set\n",
    "        X_valid_tensor = torch.tensor(X_valid,device=device).float()\n",
    "        Y_valid_tensor = torch.tensor(Y_valid,dtype=torch.long,device=device)\n",
    "        \n",
    "        # calculate validation metrics to keep track of progress; don't need predictions now\n",
    "        valid_loss, valid_acc, _ = validate(X_valid_tensor,Y_valid_tensor)\n",
    "        \n",
    "        # accumulate scalar performance metrics at each epoch to track and plot later\n",
    "        train_losses.append(epoch_loss)\n",
    "        valid_losses.append(valid_loss)\n",
    "                  \n",
    "        # Save checkpoint of the model\n",
    "        checkpoint_filename = '/content/gdrive/My Drive/DL/models/checkpoints/parallel_all_you_wantFINAL-{:03d}.pkl'.format(epoch)\n",
    "        save_checkpoint(optimizer, model, epoch, checkpoint_filename)\n",
    "        \n",
    "        # keep track of each epoch's progress\n",
    "        print(f'\\nEpoch {epoch} --- loss:{epoch_loss:.3f}, Epoch accuracy:{epoch_acc:.2f}%, Validation loss:{valid_loss:.3f}, Validation accuracy:{valid_acc:.2f}%')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "sie3lrs3XkRf",
    "outputId": "22e9ac29-ceb3-4f42-c989-d81c3f650619"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0: iteration 106/107\n",
      "Epoch 0 --- loss:3.199, Epoch accuracy:22.55%, Validation loss:2.501, Validation accuracy:13.75%\n",
      "Epoch 1: iteration 106/107\n",
      "Epoch 1 --- loss:1.804, Epoch accuracy:31.10%, Validation loss:2.059, Validation accuracy:23.31%\n",
      "Epoch 2: iteration 106/107\n",
      "Epoch 2 --- loss:1.727, Epoch accuracy:34.09%, Validation loss:2.031, Validation accuracy:27.04%\n",
      "Epoch 3: iteration 106/107\n",
      "Epoch 3 --- loss:1.658, Epoch accuracy:37.11%, Validation loss:1.962, Validation accuracy:26.34%\n",
      "Epoch 4: iteration 106/107\n",
      "Epoch 4 --- loss:1.542, Epoch accuracy:41.88%, Validation loss:1.722, Validation accuracy:38.00%\n",
      "Epoch 5: iteration 106/107\n",
      "Epoch 5 --- loss:1.485, Epoch accuracy:44.61%, Validation loss:1.616, Validation accuracy:42.89%\n",
      "Epoch 6: iteration 106/107\n",
      "Epoch 6 --- loss:1.406, Epoch accuracy:47.40%, Validation loss:1.482, Validation accuracy:47.32%\n",
      "Epoch 7: iteration 106/107\n",
      "Epoch 7 --- loss:1.337, Epoch accuracy:49.43%, Validation loss:1.474, Validation accuracy:51.52%\n",
      "Epoch 8: iteration 106/107\n",
      "Epoch 8 --- loss:1.293, Epoch accuracy:50.77%, Validation loss:1.488, Validation accuracy:46.85%\n",
      "Epoch 9: iteration 106/107\n",
      "Epoch 9 --- loss:1.226, Epoch accuracy:53.73%, Validation loss:1.390, Validation accuracy:51.98%\n",
      "Epoch 10: iteration 106/107\n",
      "Epoch 10 --- loss:1.184, Epoch accuracy:56.67%, Validation loss:1.251, Validation accuracy:55.24%\n",
      "Epoch 11: iteration 106/107\n",
      "Epoch 11 --- loss:1.161, Epoch accuracy:57.40%, Validation loss:1.193, Validation accuracy:53.61%\n",
      "Epoch 12: iteration 106/107\n",
      "Epoch 12 --- loss:1.094, Epoch accuracy:59.20%, Validation loss:1.195, Validation accuracy:53.61%\n",
      "Epoch 13: iteration 106/107\n",
      "Epoch 13 --- loss:1.079, Epoch accuracy:60.22%, Validation loss:1.241, Validation accuracy:56.18%\n",
      "Epoch 14: iteration 106/107\n",
      "Epoch 14 --- loss:1.047, Epoch accuracy:61.99%, Validation loss:1.205, Validation accuracy:55.48%\n",
      "Epoch 15: iteration 106/107\n",
      "Epoch 15 --- loss:1.002, Epoch accuracy:62.89%, Validation loss:1.189, Validation accuracy:58.04%\n",
      "Epoch 16: iteration 106/107\n",
      "Epoch 16 --- loss:0.963, Epoch accuracy:64.49%, Validation loss:1.096, Validation accuracy:61.31%\n",
      "Epoch 17: iteration 106/107\n",
      "Epoch 17 --- loss:0.946, Epoch accuracy:65.21%, Validation loss:1.188, Validation accuracy:55.71%\n",
      "Epoch 18: iteration 106/107\n",
      "Epoch 18 --- loss:0.911, Epoch accuracy:66.41%, Validation loss:1.046, Validation accuracy:62.94%\n",
      "Epoch 19: iteration 106/107\n",
      "Epoch 19 --- loss:0.901, Epoch accuracy:66.41%, Validation loss:1.174, Validation accuracy:56.41%\n",
      "Epoch 20: iteration 106/107\n",
      "Epoch 20 --- loss:0.891, Epoch accuracy:67.63%, Validation loss:1.051, Validation accuracy:60.84%\n",
      "Epoch 21: iteration 106/107\n",
      "Epoch 21 --- loss:0.850, Epoch accuracy:68.61%, Validation loss:1.233, Validation accuracy:57.81%\n",
      "Epoch 22: iteration 106/107\n",
      "Epoch 22 --- loss:0.829, Epoch accuracy:69.14%, Validation loss:1.065, Validation accuracy:64.57%\n",
      "Epoch 23: iteration 106/107\n",
      "Epoch 23 --- loss:0.807, Epoch accuracy:71.20%, Validation loss:0.965, Validation accuracy:65.97%\n",
      "Epoch 24: iteration 106/107\n",
      "Epoch 24 --- loss:0.788, Epoch accuracy:71.37%, Validation loss:1.015, Validation accuracy:65.03%\n",
      "Epoch 25: iteration 106/107\n",
      "Epoch 25 --- loss:0.772, Epoch accuracy:71.78%, Validation loss:1.028, Validation accuracy:64.34%\n",
      "Epoch 26: iteration 106/107\n",
      "Epoch 26 --- loss:0.766, Epoch accuracy:72.80%, Validation loss:1.022, Validation accuracy:64.57%\n",
      "Epoch 27: iteration 106/107\n",
      "Epoch 27 --- loss:0.730, Epoch accuracy:72.83%, Validation loss:1.058, Validation accuracy:61.54%\n",
      "Epoch 28: iteration 106/107\n",
      "Epoch 28 --- loss:0.704, Epoch accuracy:74.11%, Validation loss:1.004, Validation accuracy:64.57%\n",
      "Epoch 29: iteration 106/107\n",
      "Epoch 29 --- loss:0.687, Epoch accuracy:74.51%, Validation loss:0.956, Validation accuracy:65.27%\n",
      "Epoch 30: iteration 106/107\n",
      "Epoch 30 --- loss:0.659, Epoch accuracy:76.05%, Validation loss:0.949, Validation accuracy:67.37%\n",
      "Epoch 31: iteration 106/107\n",
      "Epoch 31 --- loss:0.644, Epoch accuracy:76.75%, Validation loss:0.967, Validation accuracy:68.07%\n",
      "Epoch 32: iteration 106/107\n",
      "Epoch 32 --- loss:0.641, Epoch accuracy:76.69%, Validation loss:0.914, Validation accuracy:64.80%\n",
      "Epoch 33: iteration 106/107\n",
      "Epoch 33 --- loss:0.623, Epoch accuracy:78.29%, Validation loss:0.991, Validation accuracy:65.27%\n",
      "Epoch 34: iteration 106/107\n",
      "Epoch 34 --- loss:0.609, Epoch accuracy:77.68%, Validation loss:0.909, Validation accuracy:66.67%\n",
      "Epoch 35: iteration 106/107\n",
      "Epoch 35 --- loss:0.591, Epoch accuracy:78.44%, Validation loss:1.072, Validation accuracy:65.50%\n",
      "Epoch 36: iteration 106/107\n",
      "Epoch 36 --- loss:0.593, Epoch accuracy:78.81%, Validation loss:0.950, Validation accuracy:64.80%\n",
      "Epoch 37: iteration 106/107\n",
      "Epoch 37 --- loss:0.552, Epoch accuracy:80.59%, Validation loss:0.899, Validation accuracy:67.13%\n",
      "Epoch 38: iteration 106/107\n",
      "Epoch 38 --- loss:0.553, Epoch accuracy:80.73%, Validation loss:0.871, Validation accuracy:68.07%\n",
      "Epoch 39: iteration 106/107\n",
      "Epoch 39 --- loss:0.542, Epoch accuracy:80.30%, Validation loss:0.892, Validation accuracy:68.53%\n",
      "Epoch 40: iteration 106/107\n",
      "Epoch 40 --- loss:0.522, Epoch accuracy:81.58%, Validation loss:0.848, Validation accuracy:68.30%\n",
      "Epoch 41: iteration 106/107\n",
      "Epoch 41 --- loss:0.520, Epoch accuracy:81.26%, Validation loss:0.920, Validation accuracy:67.83%\n",
      "Epoch 42: iteration 106/107\n",
      "Epoch 42 --- loss:0.513, Epoch accuracy:81.02%, Validation loss:0.862, Validation accuracy:66.20%\n",
      "Epoch 43: iteration 106/107\n",
      "Epoch 43 --- loss:0.477, Epoch accuracy:83.17%, Validation loss:0.881, Validation accuracy:69.00%\n",
      "Epoch 44: iteration 106/107\n",
      "Epoch 44 --- loss:0.481, Epoch accuracy:82.80%, Validation loss:0.977, Validation accuracy:66.67%\n",
      "Epoch 45: iteration 106/107\n",
      "Epoch 45 --- loss:0.486, Epoch accuracy:82.59%, Validation loss:0.959, Validation accuracy:65.27%\n",
      "Epoch 46: iteration 106/107\n",
      "Epoch 46 --- loss:0.461, Epoch accuracy:82.91%, Validation loss:0.830, Validation accuracy:66.90%\n",
      "Epoch 47: iteration 106/107\n",
      "Epoch 47 --- loss:0.441, Epoch accuracy:83.38%, Validation loss:0.926, Validation accuracy:69.00%\n",
      "Epoch 48: iteration 106/107\n",
      "Epoch 48 --- loss:0.441, Epoch accuracy:84.37%, Validation loss:1.104, Validation accuracy:62.70%\n",
      "Epoch 49: iteration 106/107\n",
      "Epoch 49 --- loss:0.442, Epoch accuracy:84.10%, Validation loss:0.810, Validation accuracy:70.40%\n",
      "Epoch 50: iteration 106/107\n",
      "Epoch 50 --- loss:0.434, Epoch accuracy:84.66%, Validation loss:0.861, Validation accuracy:68.53%\n",
      "Epoch 51: iteration 106/107\n",
      "Epoch 51 --- loss:0.436, Epoch accuracy:84.22%, Validation loss:0.826, Validation accuracy:69.70%\n",
      "Epoch 52: iteration 106/107\n",
      "Epoch 52 --- loss:0.446, Epoch accuracy:84.28%, Validation loss:0.859, Validation accuracy:69.23%\n",
      "Epoch 53: iteration 106/107\n",
      "Epoch 53 --- loss:0.415, Epoch accuracy:85.56%, Validation loss:0.855, Validation accuracy:67.13%\n",
      "Epoch 54: iteration 106/107\n",
      "Epoch 54 --- loss:0.385, Epoch accuracy:86.37%, Validation loss:0.912, Validation accuracy:66.20%\n",
      "Epoch 55: iteration 106/107\n",
      "Epoch 55 --- loss:0.387, Epoch accuracy:86.98%, Validation loss:0.848, Validation accuracy:66.90%\n",
      "Epoch 56: iteration 106/107\n",
      "Epoch 56 --- loss:0.386, Epoch accuracy:86.37%, Validation loss:0.868, Validation accuracy:69.93%\n",
      "Epoch 57: iteration 106/107\n",
      "Epoch 57 --- loss:0.371, Epoch accuracy:87.01%, Validation loss:0.807, Validation accuracy:71.79%\n",
      "Epoch 58: iteration 106/107\n",
      "Epoch 58 --- loss:0.367, Epoch accuracy:87.04%, Validation loss:0.888, Validation accuracy:69.00%\n",
      "Epoch 59: iteration 106/107\n",
      "Epoch 59 --- loss:0.356, Epoch accuracy:87.59%, Validation loss:0.886, Validation accuracy:68.53%\n",
      "Epoch 60: iteration 106/107\n",
      "Epoch 60 --- loss:0.341, Epoch accuracy:88.14%, Validation loss:0.880, Validation accuracy:70.40%\n",
      "Epoch 61: iteration 106/107\n",
      "Epoch 61 --- loss:0.337, Epoch accuracy:88.32%, Validation loss:0.844, Validation accuracy:71.10%\n",
      "Epoch 62: iteration 106/107\n",
      "Epoch 62 --- loss:0.346, Epoch accuracy:88.11%, Validation loss:0.846, Validation accuracy:71.56%\n",
      "Epoch 63: iteration 106/107\n",
      "Epoch 63 --- loss:0.332, Epoch accuracy:88.58%, Validation loss:0.897, Validation accuracy:68.30%\n",
      "Epoch 64: iteration 106/107\n",
      "Epoch 64 --- loss:0.342, Epoch accuracy:88.20%, Validation loss:0.808, Validation accuracy:69.23%\n",
      "Epoch 65: iteration 106/107\n",
      "Epoch 65 --- loss:0.326, Epoch accuracy:88.40%, Validation loss:0.825, Validation accuracy:68.53%\n",
      "Epoch 66: iteration 106/107\n",
      "Epoch 66 --- loss:0.338, Epoch accuracy:88.26%, Validation loss:0.831, Validation accuracy:69.70%\n",
      "Epoch 67: iteration 106/107\n",
      "Epoch 67 --- loss:0.322, Epoch accuracy:88.32%, Validation loss:0.860, Validation accuracy:69.93%\n",
      "Epoch 68: iteration 106/107\n",
      "Epoch 68 --- loss:0.315, Epoch accuracy:89.48%, Validation loss:0.810, Validation accuracy:70.40%\n",
      "Epoch 69: iteration 106/107\n",
      "Epoch 69 --- loss:0.313, Epoch accuracy:89.07%, Validation loss:0.810, Validation accuracy:72.03%\n",
      "Epoch 70: iteration 106/107\n",
      "Epoch 70 --- loss:0.336, Epoch accuracy:88.17%, Validation loss:0.822, Validation accuracy:72.49%\n",
      "Epoch 71: iteration 106/107\n",
      "Epoch 71 --- loss:0.310, Epoch accuracy:88.96%, Validation loss:0.812, Validation accuracy:69.93%\n",
      "Epoch 72: iteration 106/107\n",
      "Epoch 72 --- loss:0.308, Epoch accuracy:89.60%, Validation loss:0.793, Validation accuracy:70.16%\n",
      "Epoch 73: iteration 106/107\n",
      "Epoch 73 --- loss:0.305, Epoch accuracy:89.77%, Validation loss:0.818, Validation accuracy:70.86%\n",
      "Epoch 74: iteration 106/107\n",
      "Epoch 74 --- loss:0.293, Epoch accuracy:89.63%, Validation loss:0.830, Validation accuracy:69.46%\n",
      "Epoch 75: iteration 106/107\n",
      "Epoch 75 --- loss:0.277, Epoch accuracy:90.26%, Validation loss:0.797, Validation accuracy:71.79%\n",
      "Epoch 76: iteration 106/107\n",
      "Epoch 76 --- loss:0.290, Epoch accuracy:90.00%, Validation loss:0.782, Validation accuracy:70.86%\n",
      "Epoch 77: iteration 106/107\n",
      "Epoch 77 --- loss:0.294, Epoch accuracy:89.19%, Validation loss:0.750, Validation accuracy:74.13%\n",
      "Epoch 78: iteration 106/107\n",
      "Epoch 78 --- loss:0.304, Epoch accuracy:89.19%, Validation loss:0.803, Validation accuracy:73.43%\n",
      "Epoch 79: iteration 106/107\n",
      "Epoch 79 --- loss:0.278, Epoch accuracy:90.29%, Validation loss:0.800, Validation accuracy:71.10%\n",
      "Epoch 80: iteration 106/107\n",
      "Epoch 80 --- loss:0.263, Epoch accuracy:90.73%, Validation loss:0.739, Validation accuracy:74.59%\n",
      "Epoch 81: iteration 106/107\n",
      "Epoch 81 --- loss:0.298, Epoch accuracy:89.60%, Validation loss:0.826, Validation accuracy:72.26%\n",
      "Epoch 82: iteration 106/107\n",
      "Epoch 82 --- loss:0.268, Epoch accuracy:90.85%, Validation loss:0.780, Validation accuracy:72.03%\n",
      "Epoch 83: iteration 106/107\n",
      "Epoch 83 --- loss:0.263, Epoch accuracy:91.17%, Validation loss:0.824, Validation accuracy:71.33%\n",
      "Epoch 84: iteration 106/107\n",
      "Epoch 84 --- loss:0.269, Epoch accuracy:90.53%, Validation loss:0.738, Validation accuracy:73.89%\n",
      "Epoch 85: iteration 106/107\n",
      "Epoch 85 --- loss:0.259, Epoch accuracy:91.46%, Validation loss:0.798, Validation accuracy:73.43%\n",
      "Epoch 86: iteration 106/107\n",
      "Epoch 86 --- loss:0.257, Epoch accuracy:90.87%, Validation loss:0.747, Validation accuracy:71.79%\n",
      "Epoch 87: iteration 106/107\n",
      "Epoch 87 --- loss:0.271, Epoch accuracy:90.29%, Validation loss:0.739, Validation accuracy:72.49%\n",
      "Epoch 88: iteration 106/107\n",
      "Epoch 88 --- loss:0.278, Epoch accuracy:90.24%, Validation loss:0.856, Validation accuracy:72.49%\n",
      "Epoch 89: iteration 106/107\n",
      "Epoch 89 --- loss:0.259, Epoch accuracy:91.25%, Validation loss:0.757, Validation accuracy:71.79%\n",
      "Epoch 90: iteration 106/107\n",
      "Epoch 90 --- loss:0.238, Epoch accuracy:91.95%, Validation loss:0.767, Validation accuracy:72.26%\n",
      "Epoch 91: iteration 106/107\n",
      "Epoch 91 --- loss:0.245, Epoch accuracy:91.49%, Validation loss:0.759, Validation accuracy:73.89%\n",
      "Epoch 92: iteration 106/107\n",
      "Epoch 92 --- loss:0.249, Epoch accuracy:91.46%, Validation loss:0.754, Validation accuracy:73.43%\n",
      "Epoch 93: iteration 106/107\n",
      "Epoch 93 --- loss:0.273, Epoch accuracy:89.94%, Validation loss:0.765, Validation accuracy:72.73%\n",
      "Epoch 94: iteration 106/107\n",
      "Epoch 94 --- loss:0.253, Epoch accuracy:91.89%, Validation loss:0.731, Validation accuracy:72.49%\n",
      "Epoch 95: iteration 106/107\n",
      "Epoch 95 --- loss:0.240, Epoch accuracy:91.60%, Validation loss:0.756, Validation accuracy:73.89%\n",
      "Epoch 96: iteration 106/107\n",
      "Epoch 96 --- loss:0.258, Epoch accuracy:91.22%, Validation loss:0.772, Validation accuracy:71.79%\n",
      "Epoch 97: iteration 106/107\n",
      "Epoch 97 --- loss:0.250, Epoch accuracy:91.46%, Validation loss:0.767, Validation accuracy:73.43%\n",
      "Epoch 98: iteration 106/107\n",
      "Epoch 98 --- loss:0.242, Epoch accuracy:91.86%, Validation loss:0.842, Validation accuracy:71.56%\n",
      "Epoch 99: iteration 106/107\n",
      "Epoch 99 --- loss:0.227, Epoch accuracy:92.18%, Validation loss:0.732, Validation accuracy:72.49%\n",
      "Epoch 100: iteration 106/107\n",
      "Epoch 100 --- loss:0.232, Epoch accuracy:91.98%, Validation loss:0.773, Validation accuracy:72.03%\n",
      "Epoch 101: iteration 106/107\n",
      "Epoch 101 --- loss:0.237, Epoch accuracy:91.80%, Validation loss:0.754, Validation accuracy:71.56%\n",
      "Epoch 102: iteration 106/107\n",
      "Epoch 102 --- loss:0.236, Epoch accuracy:92.10%, Validation loss:0.804, Validation accuracy:72.96%\n",
      "Epoch 103: iteration 106/107\n",
      "Epoch 103 --- loss:0.221, Epoch accuracy:92.56%, Validation loss:0.754, Validation accuracy:73.89%\n",
      "Epoch 104: iteration 106/107\n",
      "Epoch 104 --- loss:0.222, Epoch accuracy:92.44%, Validation loss:0.769, Validation accuracy:72.03%\n",
      "Epoch 105: iteration 106/107\n",
      "Epoch 105 --- loss:0.229, Epoch accuracy:92.21%, Validation loss:0.839, Validation accuracy:70.16%\n",
      "Epoch 106: iteration 106/107\n",
      "Epoch 106 --- loss:0.225, Epoch accuracy:91.95%, Validation loss:0.793, Validation accuracy:72.26%\n",
      "Epoch 107: iteration 106/107\n",
      "Epoch 107 --- loss:0.230, Epoch accuracy:92.39%, Validation loss:0.756, Validation accuracy:72.49%\n",
      "Epoch 108: iteration 106/107\n",
      "Epoch 108 --- loss:0.211, Epoch accuracy:93.23%, Validation loss:0.747, Validation accuracy:72.26%\n",
      "Epoch 109: iteration 106/107\n",
      "Epoch 109 --- loss:0.218, Epoch accuracy:92.36%, Validation loss:0.770, Validation accuracy:74.13%\n",
      "Epoch 110: iteration 106/107\n",
      "Epoch 110 --- loss:0.226, Epoch accuracy:92.71%, Validation loss:0.759, Validation accuracy:70.86%\n",
      "Epoch 111: iteration 106/107\n",
      "Epoch 111 --- loss:0.224, Epoch accuracy:92.07%, Validation loss:0.808, Validation accuracy:71.56%\n",
      "Epoch 112: iteration 106/107\n",
      "Epoch 112 --- loss:0.218, Epoch accuracy:92.65%, Validation loss:0.813, Validation accuracy:70.86%\n",
      "Epoch 113: iteration 106/107\n",
      "Epoch 113 --- loss:0.228, Epoch accuracy:91.78%, Validation loss:0.794, Validation accuracy:71.33%\n",
      "Epoch 114: iteration 106/107\n",
      "Epoch 114 --- loss:0.212, Epoch accuracy:93.00%, Validation loss:0.814, Validation accuracy:71.33%\n",
      "Epoch 115: iteration 106/107\n",
      "Epoch 115 --- loss:0.217, Epoch accuracy:92.42%, Validation loss:0.797, Validation accuracy:71.79%\n",
      "Epoch 116: iteration 106/107\n",
      "Epoch 116 --- loss:0.186, Epoch accuracy:93.66%, Validation loss:0.750, Validation accuracy:71.79%\n",
      "Epoch 117: iteration 106/107\n",
      "Epoch 117 --- loss:0.210, Epoch accuracy:92.88%, Validation loss:0.743, Validation accuracy:72.03%\n",
      "Epoch 118: iteration 106/107\n",
      "Epoch 118 --- loss:0.233, Epoch accuracy:91.63%, Validation loss:0.847, Validation accuracy:69.93%\n",
      "Epoch 119: iteration 106/107\n",
      "Epoch 119 --- loss:0.211, Epoch accuracy:92.94%, Validation loss:0.763, Validation accuracy:71.10%\n",
      "Epoch 120: iteration 106/107\n",
      "Epoch 120 --- loss:0.207, Epoch accuracy:93.08%, Validation loss:0.795, Validation accuracy:70.40%\n",
      "Epoch 121: iteration 106/107\n",
      "Epoch 121 --- loss:0.192, Epoch accuracy:93.84%, Validation loss:0.819, Validation accuracy:72.26%\n",
      "Epoch 122: iteration 106/107\n",
      "Epoch 122 --- loss:0.220, Epoch accuracy:92.44%, Validation loss:0.809, Validation accuracy:70.86%\n",
      "Epoch 123: iteration 106/107\n",
      "Epoch 123 --- loss:0.211, Epoch accuracy:92.53%, Validation loss:0.806, Validation accuracy:70.63%\n",
      "Epoch 124: iteration 106/107\n",
      "Epoch 124 --- loss:0.210, Epoch accuracy:92.53%, Validation loss:0.763, Validation accuracy:72.73%\n",
      "Epoch 125: iteration 106/107\n",
      "Epoch 125 --- loss:0.197, Epoch accuracy:93.58%, Validation loss:0.750, Validation accuracy:70.40%\n",
      "Epoch 126: iteration 106/107\n",
      "Epoch 126 --- loss:0.197, Epoch accuracy:93.29%, Validation loss:0.781, Validation accuracy:71.79%\n",
      "Epoch 127: iteration 106/107\n",
      "Epoch 127 --- loss:0.205, Epoch accuracy:93.11%, Validation loss:0.834, Validation accuracy:71.33%\n",
      "Epoch 128: iteration 106/107\n",
      "Epoch 128 --- loss:0.200, Epoch accuracy:93.17%, Validation loss:0.763, Validation accuracy:72.49%\n",
      "Epoch 129: iteration 106/107\n",
      "Epoch 129 --- loss:0.207, Epoch accuracy:93.32%, Validation loss:0.798, Validation accuracy:72.26%\n",
      "Epoch 130: iteration 106/107\n",
      "Epoch 130 --- loss:0.225, Epoch accuracy:92.62%, Validation loss:0.752, Validation accuracy:74.59%\n",
      "Epoch 131: iteration 106/107\n",
      "Epoch 131 --- loss:0.203, Epoch accuracy:93.40%, Validation loss:0.782, Validation accuracy:73.66%\n",
      "Epoch 132: iteration 106/107\n",
      "Epoch 132 --- loss:0.209, Epoch accuracy:93.20%, Validation loss:0.792, Validation accuracy:70.86%\n",
      "Epoch 133: iteration 106/107\n",
      "Epoch 133 --- loss:0.198, Epoch accuracy:93.58%, Validation loss:0.763, Validation accuracy:69.00%\n",
      "Epoch 134: iteration 106/107\n",
      "Epoch 134 --- loss:0.212, Epoch accuracy:93.20%, Validation loss:0.798, Validation accuracy:71.10%\n",
      "Epoch 135: iteration 106/107\n",
      "Epoch 135 --- loss:0.197, Epoch accuracy:93.26%, Validation loss:0.798, Validation accuracy:70.40%\n",
      "Epoch 136: iteration 106/107\n",
      "Epoch 136 --- loss:0.186, Epoch accuracy:93.72%, Validation loss:0.800, Validation accuracy:70.40%\n",
      "Epoch 137: iteration 106/107\n",
      "Epoch 137 --- loss:0.203, Epoch accuracy:92.85%, Validation loss:0.814, Validation accuracy:71.33%\n",
      "Epoch 138: iteration 106/107\n",
      "Epoch 138 --- loss:0.201, Epoch accuracy:92.97%, Validation loss:0.798, Validation accuracy:68.76%\n",
      "Epoch 139: iteration 106/107\n",
      "Epoch 139 --- loss:0.188, Epoch accuracy:93.55%, Validation loss:0.771, Validation accuracy:71.10%\n",
      "Epoch 140: iteration 106/107\n",
      "Epoch 140 --- loss:0.213, Epoch accuracy:92.04%, Validation loss:0.845, Validation accuracy:71.33%\n",
      "Epoch 141: iteration 106/107\n",
      "Epoch 141 --- loss:0.184, Epoch accuracy:94.01%, Validation loss:0.772, Validation accuracy:70.63%\n",
      "Epoch 142: iteration 106/107\n",
      "Epoch 142 --- loss:0.204, Epoch accuracy:92.91%, Validation loss:0.764, Validation accuracy:71.33%\n",
      "Epoch 143: iteration 106/107\n",
      "Epoch 143 --- loss:0.201, Epoch accuracy:92.85%, Validation loss:0.811, Validation accuracy:69.00%\n",
      "Epoch 144: iteration 106/107\n",
      "Epoch 144 --- loss:0.192, Epoch accuracy:93.35%, Validation loss:0.754, Validation accuracy:69.93%\n",
      "Epoch 145: iteration 106/107\n",
      "Epoch 145 --- loss:0.190, Epoch accuracy:93.93%, Validation loss:0.743, Validation accuracy:72.73%\n",
      "Epoch 146: iteration 106/107\n",
      "Epoch 146 --- loss:0.195, Epoch accuracy:93.64%, Validation loss:0.766, Validation accuracy:71.79%\n",
      "Epoch 147: iteration 106/107\n",
      "Epoch 147 --- loss:0.186, Epoch accuracy:94.04%, Validation loss:0.744, Validation accuracy:71.33%\n",
      "Epoch 148: iteration 106/107\n",
      "Epoch 148 --- loss:0.196, Epoch accuracy:93.46%, Validation loss:0.801, Validation accuracy:68.53%\n",
      "Epoch 149: iteration 106/107\n",
      "Epoch 149 --- loss:0.176, Epoch accuracy:93.98%, Validation loss:0.713, Validation accuracy:72.49%\n",
      "Epoch 150: iteration 106/107\n",
      "Epoch 150 --- loss:0.189, Epoch accuracy:93.78%, Validation loss:0.751, Validation accuracy:71.33%\n",
      "Epoch 151: iteration 106/107\n",
      "Epoch 151 --- loss:0.169, Epoch accuracy:94.71%, Validation loss:0.831, Validation accuracy:70.86%\n",
      "Epoch 152: iteration 106/107\n",
      "Epoch 152 --- loss:0.196, Epoch accuracy:93.17%, Validation loss:0.812, Validation accuracy:71.33%\n",
      "Epoch 153: iteration 106/107\n",
      "Epoch 153 --- loss:0.193, Epoch accuracy:93.72%, Validation loss:0.798, Validation accuracy:71.33%\n",
      "Epoch 154: iteration 106/107\n",
      "Epoch 154 --- loss:0.197, Epoch accuracy:93.49%, Validation loss:0.760, Validation accuracy:72.96%\n",
      "Epoch 155: iteration 106/107\n",
      "Epoch 155 --- loss:0.175, Epoch accuracy:94.59%, Validation loss:0.760, Validation accuracy:74.13%\n",
      "Epoch 156: iteration 106/107\n",
      "Epoch 156 --- loss:0.189, Epoch accuracy:93.72%, Validation loss:0.828, Validation accuracy:71.79%\n",
      "Epoch 157: iteration 106/107\n",
      "Epoch 157 --- loss:0.187, Epoch accuracy:93.72%, Validation loss:0.745, Validation accuracy:73.19%\n",
      "Epoch 158: iteration 106/107\n",
      "Epoch 158 --- loss:0.191, Epoch accuracy:93.37%, Validation loss:0.711, Validation accuracy:74.59%\n",
      "Epoch 159: iteration 106/107\n",
      "Epoch 159 --- loss:0.183, Epoch accuracy:94.10%, Validation loss:0.809, Validation accuracy:71.10%\n",
      "Epoch 160: iteration 106/107\n",
      "Epoch 160 --- loss:0.179, Epoch accuracy:93.87%, Validation loss:0.798, Validation accuracy:72.96%\n",
      "Epoch 161: iteration 106/107\n",
      "Epoch 161 --- loss:0.181, Epoch accuracy:94.07%, Validation loss:0.750, Validation accuracy:73.43%\n",
      "Epoch 162: iteration 106/107\n",
      "Epoch 162 --- loss:0.186, Epoch accuracy:93.78%, Validation loss:0.837, Validation accuracy:71.79%\n",
      "Epoch 163: iteration 106/107\n",
      "Epoch 163 --- loss:0.205, Epoch accuracy:93.43%, Validation loss:0.762, Validation accuracy:73.66%\n",
      "Epoch 164: iteration 106/107\n",
      "Epoch 164 --- loss:0.180, Epoch accuracy:94.16%, Validation loss:0.830, Validation accuracy:71.79%\n",
      "Epoch 165: iteration 106/107\n",
      "Epoch 165 --- loss:0.188, Epoch accuracy:93.84%, Validation loss:0.772, Validation accuracy:74.36%\n",
      "Epoch 166: iteration 106/107\n",
      "Epoch 166 --- loss:0.175, Epoch accuracy:94.04%, Validation loss:0.728, Validation accuracy:75.52%\n",
      "Epoch 167: iteration 106/107\n",
      "Epoch 167 --- loss:0.174, Epoch accuracy:94.45%, Validation loss:0.837, Validation accuracy:72.49%\n",
      "Epoch 168: iteration 106/107\n",
      "Epoch 168 --- loss:0.161, Epoch accuracy:94.77%, Validation loss:0.764, Validation accuracy:74.13%\n",
      "Epoch 169: iteration 106/107\n",
      "Epoch 169 --- loss:0.168, Epoch accuracy:94.13%, Validation loss:0.827, Validation accuracy:73.43%\n",
      "Epoch 170: iteration 106/107\n",
      "Epoch 170 --- loss:0.175, Epoch accuracy:94.45%, Validation loss:0.753, Validation accuracy:73.89%\n",
      "Epoch 171: iteration 106/107\n",
      "Epoch 171 --- loss:0.176, Epoch accuracy:94.22%, Validation loss:0.756, Validation accuracy:73.89%\n",
      "Epoch 172: iteration 106/107\n",
      "Epoch 172 --- loss:0.180, Epoch accuracy:93.93%, Validation loss:0.755, Validation accuracy:73.19%\n",
      "Epoch 173: iteration 106/107\n",
      "Epoch 173 --- loss:0.189, Epoch accuracy:93.87%, Validation loss:0.765, Validation accuracy:75.06%\n",
      "Epoch 174: iteration 106/107\n",
      "Epoch 174 --- loss:0.161, Epoch accuracy:94.94%, Validation loss:0.716, Validation accuracy:73.43%\n",
      "Epoch 175: iteration 106/107\n",
      "Epoch 175 --- loss:0.174, Epoch accuracy:93.75%, Validation loss:0.782, Validation accuracy:72.73%\n",
      "Epoch 176: iteration 106/107\n",
      "Epoch 176 --- loss:0.158, Epoch accuracy:95.09%, Validation loss:0.728, Validation accuracy:74.59%\n",
      "Epoch 177: iteration 106/107\n",
      "Epoch 177 --- loss:0.181, Epoch accuracy:94.25%, Validation loss:0.705, Validation accuracy:74.59%\n",
      "Epoch 178: iteration 106/107\n",
      "Epoch 178 --- loss:0.182, Epoch accuracy:93.87%, Validation loss:0.757, Validation accuracy:73.19%\n",
      "Epoch 179: iteration 106/107\n",
      "Epoch 179 --- loss:0.183, Epoch accuracy:93.98%, Validation loss:0.759, Validation accuracy:74.36%\n",
      "Epoch 180: iteration 106/107\n",
      "Epoch 180 --- loss:0.185, Epoch accuracy:94.13%, Validation loss:0.787, Validation accuracy:72.73%\n",
      "Epoch 181: iteration 106/107\n",
      "Epoch 181 --- loss:0.182, Epoch accuracy:93.81%, Validation loss:0.772, Validation accuracy:72.96%\n",
      "Epoch 182: iteration 106/107\n",
      "Epoch 182 --- loss:0.186, Epoch accuracy:94.10%, Validation loss:0.825, Validation accuracy:72.49%\n",
      "Epoch 183: iteration 106/107\n",
      "Epoch 183 --- loss:0.194, Epoch accuracy:93.14%, Validation loss:0.795, Validation accuracy:70.16%\n",
      "Epoch 184: iteration 106/107\n",
      "Epoch 184 --- loss:0.173, Epoch accuracy:94.22%, Validation loss:0.761, Validation accuracy:75.06%\n",
      "Epoch 185: iteration 106/107\n",
      "Epoch 185 --- loss:0.183, Epoch accuracy:94.39%, Validation loss:0.710, Validation accuracy:75.06%\n",
      "Epoch 186: iteration 106/107\n",
      "Epoch 186 --- loss:0.167, Epoch accuracy:94.45%, Validation loss:0.729, Validation accuracy:73.19%\n",
      "Epoch 187: iteration 106/107\n",
      "Epoch 187 --- loss:0.160, Epoch accuracy:94.94%, Validation loss:0.746, Validation accuracy:76.22%\n",
      "Epoch 188: iteration 106/107\n",
      "Epoch 188 --- loss:0.176, Epoch accuracy:94.25%, Validation loss:0.811, Validation accuracy:73.66%\n",
      "Epoch 189: iteration 106/107\n",
      "Epoch 189 --- loss:0.155, Epoch accuracy:95.38%, Validation loss:0.798, Validation accuracy:73.19%\n",
      "Epoch 190: iteration 106/107\n",
      "Epoch 190 --- loss:0.170, Epoch accuracy:94.25%, Validation loss:0.712, Validation accuracy:75.06%\n",
      "Epoch 191: iteration 106/107\n",
      "Epoch 191 --- loss:0.183, Epoch accuracy:93.84%, Validation loss:0.747, Validation accuracy:73.66%\n",
      "Epoch 192: iteration 106/107\n",
      "Epoch 192 --- loss:0.163, Epoch accuracy:94.86%, Validation loss:0.774, Validation accuracy:73.89%\n",
      "Epoch 193: iteration 106/107\n",
      "Epoch 193 --- loss:0.168, Epoch accuracy:94.42%, Validation loss:0.802, Validation accuracy:73.66%\n",
      "Epoch 194: iteration 106/107\n",
      "Epoch 194 --- loss:0.165, Epoch accuracy:94.48%, Validation loss:0.736, Validation accuracy:76.69%\n",
      "Epoch 195: iteration 106/107\n",
      "Epoch 195 --- loss:0.172, Epoch accuracy:94.19%, Validation loss:0.784, Validation accuracy:71.56%\n",
      "Epoch 196: iteration 106/107\n",
      "Epoch 196 --- loss:0.166, Epoch accuracy:94.65%, Validation loss:0.776, Validation accuracy:72.96%\n",
      "Epoch 197: iteration 106/107\n",
      "Epoch 197 --- loss:0.166, Epoch accuracy:94.48%, Validation loss:0.795, Validation accuracy:72.49%\n",
      "Epoch 198: iteration 106/107\n",
      "Epoch 198 --- loss:0.177, Epoch accuracy:93.98%, Validation loss:0.730, Validation accuracy:73.43%\n",
      "Epoch 199: iteration 106/107\n",
      "Epoch 199 --- loss:0.164, Epoch accuracy:94.80%, Validation loss:0.726, Validation accuracy:73.19%\n",
      "Epoch 200: iteration 106/107\n",
      "Epoch 200 --- loss:0.183, Epoch accuracy:94.13%, Validation loss:0.731, Validation accuracy:73.66%\n",
      "Epoch 201: iteration 106/107\n",
      "Epoch 201 --- loss:0.160, Epoch accuracy:94.65%, Validation loss:0.718, Validation accuracy:73.43%\n",
      "Epoch 202: iteration 106/107\n",
      "Epoch 202 --- loss:0.169, Epoch accuracy:94.33%, Validation loss:0.731, Validation accuracy:73.66%\n",
      "Epoch 203: iteration 106/107\n",
      "Epoch 203 --- loss:0.175, Epoch accuracy:94.33%, Validation loss:0.708, Validation accuracy:76.46%\n",
      "Epoch 204: iteration 106/107\n",
      "Epoch 204 --- loss:0.162, Epoch accuracy:94.48%, Validation loss:0.759, Validation accuracy:75.06%\n",
      "Epoch 205: iteration 106/107\n",
      "Epoch 205 --- loss:0.151, Epoch accuracy:95.18%, Validation loss:0.688, Validation accuracy:75.76%\n",
      "Epoch 206: iteration 106/107\n",
      "Epoch 206 --- loss:0.164, Epoch accuracy:94.22%, Validation loss:0.737, Validation accuracy:72.73%\n",
      "Epoch 207: iteration 106/107\n",
      "Epoch 207 --- loss:0.170, Epoch accuracy:94.51%, Validation loss:0.724, Validation accuracy:73.19%\n",
      "Epoch 208: iteration 106/107\n",
      "Epoch 208 --- loss:0.168, Epoch accuracy:94.36%, Validation loss:0.748, Validation accuracy:74.13%\n",
      "Epoch 209: iteration 106/107\n",
      "Epoch 209 --- loss:0.168, Epoch accuracy:94.42%, Validation loss:0.712, Validation accuracy:75.76%\n",
      "Epoch 210: iteration 106/107\n",
      "Epoch 210 --- loss:0.167, Epoch accuracy:94.39%, Validation loss:0.750, Validation accuracy:75.52%\n",
      "Epoch 211: iteration 106/107\n",
      "Epoch 211 --- loss:0.168, Epoch accuracy:94.39%, Validation loss:0.720, Validation accuracy:75.29%\n",
      "Epoch 212: iteration 106/107\n",
      "Epoch 212 --- loss:0.162, Epoch accuracy:94.80%, Validation loss:0.712, Validation accuracy:75.29%\n",
      "Epoch 213: iteration 106/107\n",
      "Epoch 213 --- loss:0.160, Epoch accuracy:94.57%, Validation loss:0.728, Validation accuracy:72.96%\n",
      "Epoch 214: iteration 106/107\n",
      "Epoch 214 --- loss:0.157, Epoch accuracy:95.00%, Validation loss:0.716, Validation accuracy:74.36%\n",
      "Epoch 215: iteration 106/107\n",
      "Epoch 215 --- loss:0.168, Epoch accuracy:94.36%, Validation loss:0.760, Validation accuracy:73.43%\n",
      "Epoch 216: iteration 106/107\n",
      "Epoch 216 --- loss:0.164, Epoch accuracy:94.45%, Validation loss:0.714, Validation accuracy:75.29%\n",
      "Epoch 217: iteration 106/107\n",
      "Epoch 217 --- loss:0.154, Epoch accuracy:94.97%, Validation loss:0.719, Validation accuracy:76.92%\n",
      "Epoch 218: iteration 106/107\n",
      "Epoch 218 --- loss:0.165, Epoch accuracy:94.65%, Validation loss:0.753, Validation accuracy:70.86%\n",
      "Epoch 219: iteration 106/107\n",
      "Epoch 219 --- loss:0.168, Epoch accuracy:94.36%, Validation loss:0.649, Validation accuracy:77.16%\n",
      "Epoch 220: iteration 106/107\n",
      "Epoch 220 --- loss:0.167, Epoch accuracy:94.22%, Validation loss:0.665, Validation accuracy:77.62%\n",
      "Epoch 221: iteration 106/107\n",
      "Epoch 221 --- loss:0.162, Epoch accuracy:94.77%, Validation loss:0.735, Validation accuracy:72.96%\n",
      "Epoch 222: iteration 106/107\n",
      "Epoch 222 --- loss:0.176, Epoch accuracy:93.75%, Validation loss:0.720, Validation accuracy:73.89%\n",
      "Epoch 223: iteration 106/107\n",
      "Epoch 223 --- loss:0.172, Epoch accuracy:94.42%, Validation loss:0.728, Validation accuracy:74.13%\n",
      "Epoch 224: iteration 106/107\n",
      "Epoch 224 --- loss:0.173, Epoch accuracy:94.01%, Validation loss:0.663, Validation accuracy:77.16%\n",
      "Epoch 225: iteration 106/107\n",
      "Epoch 225 --- loss:0.179, Epoch accuracy:94.04%, Validation loss:0.734, Validation accuracy:75.29%\n",
      "Epoch 226: iteration 106/107\n",
      "Epoch 226 --- loss:0.163, Epoch accuracy:94.57%, Validation loss:0.737, Validation accuracy:72.73%\n",
      "Epoch 227: iteration 106/107\n",
      "Epoch 227 --- loss:0.173, Epoch accuracy:94.07%, Validation loss:0.739, Validation accuracy:74.83%\n",
      "Epoch 228: iteration 106/107\n",
      "Epoch 228 --- loss:0.166, Epoch accuracy:94.27%, Validation loss:0.672, Validation accuracy:74.36%\n",
      "Epoch 229: iteration 106/107\n",
      "Epoch 229 --- loss:0.148, Epoch accuracy:95.00%, Validation loss:0.677, Validation accuracy:77.39%\n",
      "Epoch 230: iteration 106/107\n",
      "Epoch 230 --- loss:0.160, Epoch accuracy:94.59%, Validation loss:0.683, Validation accuracy:78.32%\n",
      "Epoch 231: iteration 106/107\n",
      "Epoch 231 --- loss:0.162, Epoch accuracy:94.57%, Validation loss:0.706, Validation accuracy:75.99%\n",
      "Epoch 232: iteration 106/107\n",
      "Epoch 232 --- loss:0.162, Epoch accuracy:94.65%, Validation loss:0.660, Validation accuracy:76.22%\n",
      "Epoch 233: iteration 106/107\n",
      "Epoch 233 --- loss:0.168, Epoch accuracy:94.42%, Validation loss:0.693, Validation accuracy:75.29%\n",
      "Epoch 234: iteration 106/107\n",
      "Epoch 234 --- loss:0.164, Epoch accuracy:94.77%, Validation loss:0.760, Validation accuracy:75.06%\n",
      "Epoch 235: iteration 106/107\n",
      "Epoch 235 --- loss:0.167, Epoch accuracy:94.59%, Validation loss:0.672, Validation accuracy:75.29%\n",
      "Epoch 236: iteration 106/107\n",
      "Epoch 236 --- loss:0.157, Epoch accuracy:94.54%, Validation loss:0.702, Validation accuracy:75.52%\n",
      "Epoch 237: iteration 106/107\n",
      "Epoch 237 --- loss:0.158, Epoch accuracy:95.00%, Validation loss:0.714, Validation accuracy:73.66%\n",
      "Epoch 238: iteration 106/107\n",
      "Epoch 238 --- loss:0.149, Epoch accuracy:95.12%, Validation loss:0.703, Validation accuracy:75.06%\n",
      "Epoch 239: iteration 106/107\n",
      "Epoch 239 --- loss:0.165, Epoch accuracy:94.57%, Validation loss:0.742, Validation accuracy:73.43%\n",
      "Epoch 240: iteration 106/107\n",
      "Epoch 240 --- loss:0.145, Epoch accuracy:94.68%, Validation loss:0.658, Validation accuracy:76.92%\n",
      "Epoch 241: iteration 106/107\n",
      "Epoch 241 --- loss:0.159, Epoch accuracy:95.03%, Validation loss:0.707, Validation accuracy:75.76%\n",
      "Epoch 242: iteration 106/107\n",
      "Epoch 242 --- loss:0.151, Epoch accuracy:95.18%, Validation loss:0.728, Validation accuracy:73.43%\n",
      "Epoch 243: iteration 106/107\n",
      "Epoch 243 --- loss:0.164, Epoch accuracy:94.22%, Validation loss:0.754, Validation accuracy:75.29%\n",
      "Epoch 244: iteration 106/107\n",
      "Epoch 244 --- loss:0.151, Epoch accuracy:95.09%, Validation loss:0.721, Validation accuracy:75.52%\n",
      "Epoch 245: iteration 106/107\n",
      "Epoch 245 --- loss:0.153, Epoch accuracy:94.89%, Validation loss:0.688, Validation accuracy:74.83%\n",
      "Epoch 246: iteration 106/107\n",
      "Epoch 246 --- loss:0.160, Epoch accuracy:94.74%, Validation loss:0.686, Validation accuracy:73.89%\n",
      "Epoch 247: iteration 106/107\n",
      "Epoch 247 --- loss:0.156, Epoch accuracy:94.97%, Validation loss:0.719, Validation accuracy:74.59%\n",
      "Epoch 248: iteration 106/107\n",
      "Epoch 248 --- loss:0.162, Epoch accuracy:94.91%, Validation loss:0.693, Validation accuracy:76.92%\n",
      "Epoch 249: iteration 106/107\n",
      "Epoch 249 --- loss:0.167, Epoch accuracy:94.19%, Validation loss:0.682, Validation accuracy:75.76%\n",
      "Epoch 250: iteration 106/107\n",
      "Epoch 250 --- loss:0.155, Epoch accuracy:94.62%, Validation loss:0.694, Validation accuracy:73.43%\n",
      "Epoch 251: iteration 106/107\n",
      "Epoch 251 --- loss:0.150, Epoch accuracy:94.74%, Validation loss:0.730, Validation accuracy:74.59%\n",
      "Epoch 252: iteration 106/107\n",
      "Epoch 252 --- loss:0.145, Epoch accuracy:95.38%, Validation loss:0.715, Validation accuracy:74.36%\n",
      "Epoch 253: iteration 106/107\n",
      "Epoch 253 --- loss:0.172, Epoch accuracy:94.33%, Validation loss:0.705, Validation accuracy:72.49%\n",
      "Epoch 254: iteration 106/107\n",
      "Epoch 254 --- loss:0.164, Epoch accuracy:94.39%, Validation loss:0.693, Validation accuracy:73.66%\n",
      "Epoch 255: iteration 106/107\n",
      "Epoch 255 --- loss:0.160, Epoch accuracy:94.77%, Validation loss:0.705, Validation accuracy:75.29%\n",
      "Epoch 256: iteration 106/107\n",
      "Epoch 256 --- loss:0.158, Epoch accuracy:94.80%, Validation loss:0.722, Validation accuracy:74.59%\n",
      "Epoch 257: iteration 106/107\n",
      "Epoch 257 --- loss:0.155, Epoch accuracy:94.80%, Validation loss:0.692, Validation accuracy:74.83%\n",
      "Epoch 258: iteration 106/107\n",
      "Epoch 258 --- loss:0.152, Epoch accuracy:95.15%, Validation loss:0.644, Validation accuracy:76.46%\n",
      "Epoch 259: iteration 106/107\n",
      "Epoch 259 --- loss:0.157, Epoch accuracy:94.71%, Validation loss:0.687, Validation accuracy:75.29%\n",
      "Epoch 260: iteration 106/107\n",
      "Epoch 260 --- loss:0.170, Epoch accuracy:94.42%, Validation loss:0.694, Validation accuracy:75.29%\n",
      "Epoch 261: iteration 106/107\n",
      "Epoch 261 --- loss:0.148, Epoch accuracy:95.32%, Validation loss:0.673, Validation accuracy:76.69%\n",
      "Epoch 262: iteration 106/107\n",
      "Epoch 262 --- loss:0.157, Epoch accuracy:94.97%, Validation loss:0.757, Validation accuracy:73.19%\n",
      "Epoch 263: iteration 106/107\n",
      "Epoch 263 --- loss:0.159, Epoch accuracy:94.74%, Validation loss:0.766, Validation accuracy:72.73%\n",
      "Epoch 264: iteration 106/107\n",
      "Epoch 264 --- loss:0.168, Epoch accuracy:94.10%, Validation loss:0.763, Validation accuracy:72.96%\n",
      "Epoch 265: iteration 106/107\n",
      "Epoch 265 --- loss:0.147, Epoch accuracy:95.32%, Validation loss:0.675, Validation accuracy:73.89%\n",
      "Epoch 266: iteration 106/107\n",
      "Epoch 266 --- loss:0.153, Epoch accuracy:95.06%, Validation loss:0.744, Validation accuracy:75.06%\n",
      "Epoch 267: iteration 106/107\n",
      "Epoch 267 --- loss:0.147, Epoch accuracy:95.00%, Validation loss:0.720, Validation accuracy:72.96%\n",
      "Epoch 268: iteration 106/107\n",
      "Epoch 268 --- loss:0.164, Epoch accuracy:94.54%, Validation loss:0.704, Validation accuracy:74.36%\n",
      "Epoch 269: iteration 106/107\n",
      "Epoch 269 --- loss:0.177, Epoch accuracy:94.04%, Validation loss:0.683, Validation accuracy:74.59%\n",
      "Epoch 270: iteration 106/107\n",
      "Epoch 270 --- loss:0.152, Epoch accuracy:95.12%, Validation loss:0.700, Validation accuracy:75.06%\n",
      "Epoch 271: iteration 106/107\n",
      "Epoch 271 --- loss:0.170, Epoch accuracy:94.01%, Validation loss:0.798, Validation accuracy:71.10%\n",
      "Epoch 272: iteration 106/107\n",
      "Epoch 272 --- loss:0.140, Epoch accuracy:95.58%, Validation loss:0.690, Validation accuracy:76.22%\n",
      "Epoch 273: iteration 106/107\n",
      "Epoch 273 --- loss:0.152, Epoch accuracy:94.65%, Validation loss:0.776, Validation accuracy:72.96%\n",
      "Epoch 274: iteration 106/107\n",
      "Epoch 274 --- loss:0.155, Epoch accuracy:94.83%, Validation loss:0.691, Validation accuracy:75.52%\n",
      "Epoch 275: iteration 106/107\n",
      "Epoch 275 --- loss:0.148, Epoch accuracy:95.23%, Validation loss:0.718, Validation accuracy:75.76%\n",
      "Epoch 276: iteration 106/107\n",
      "Epoch 276 --- loss:0.158, Epoch accuracy:94.74%, Validation loss:0.697, Validation accuracy:75.99%\n",
      "Epoch 277: iteration 106/107\n",
      "Epoch 277 --- loss:0.166, Epoch accuracy:94.59%, Validation loss:0.702, Validation accuracy:77.16%\n",
      "Epoch 278: iteration 106/107\n",
      "Epoch 278 --- loss:0.163, Epoch accuracy:94.36%, Validation loss:0.693, Validation accuracy:74.13%\n",
      "Epoch 279: iteration 106/107\n",
      "Epoch 279 --- loss:0.164, Epoch accuracy:94.86%, Validation loss:0.700, Validation accuracy:76.92%\n",
      "Epoch 280: iteration 106/107\n",
      "Epoch 280 --- loss:0.147, Epoch accuracy:95.44%, Validation loss:0.695, Validation accuracy:75.29%\n",
      "Epoch 281: iteration 106/107\n",
      "Epoch 281 --- loss:0.143, Epoch accuracy:95.20%, Validation loss:0.690, Validation accuracy:75.52%\n",
      "Epoch 282: iteration 106/107\n",
      "Epoch 282 --- loss:0.135, Epoch accuracy:95.73%, Validation loss:0.697, Validation accuracy:75.76%\n",
      "Epoch 283: iteration 106/107\n",
      "Epoch 283 --- loss:0.137, Epoch accuracy:95.47%, Validation loss:0.662, Validation accuracy:76.46%\n",
      "Epoch 284: iteration 106/107\n",
      "Epoch 284 --- loss:0.153, Epoch accuracy:94.71%, Validation loss:0.733, Validation accuracy:75.06%\n",
      "Epoch 285: iteration 106/107\n",
      "Epoch 285 --- loss:0.140, Epoch accuracy:95.06%, Validation loss:0.708, Validation accuracy:75.06%\n",
      "Epoch 286: iteration 106/107\n",
      "Epoch 286 --- loss:0.145, Epoch accuracy:95.41%, Validation loss:0.899, Validation accuracy:67.83%\n",
      "Epoch 287: iteration 106/107\n",
      "Epoch 287 --- loss:0.164, Epoch accuracy:94.30%, Validation loss:0.759, Validation accuracy:72.73%\n",
      "Epoch 288: iteration 106/107\n",
      "Epoch 288 --- loss:0.151, Epoch accuracy:95.06%, Validation loss:0.687, Validation accuracy:75.52%\n",
      "Epoch 289: iteration 106/107\n",
      "Epoch 289 --- loss:0.156, Epoch accuracy:94.74%, Validation loss:0.720, Validation accuracy:76.46%\n",
      "Epoch 290: iteration 106/107\n",
      "Epoch 290 --- loss:0.150, Epoch accuracy:94.97%, Validation loss:0.755, Validation accuracy:72.73%\n",
      "Epoch 291: iteration 106/107\n",
      "Epoch 291 --- loss:0.169, Epoch accuracy:94.45%, Validation loss:0.700, Validation accuracy:76.46%\n",
      "Epoch 292: iteration 106/107\n",
      "Epoch 292 --- loss:0.162, Epoch accuracy:94.57%, Validation loss:0.676, Validation accuracy:77.86%\n",
      "Epoch 293: iteration 106/107\n",
      "Epoch 293 --- loss:0.154, Epoch accuracy:94.89%, Validation loss:0.723, Validation accuracy:72.96%\n",
      "Epoch 294: iteration 106/107\n",
      "Epoch 294 --- loss:0.156, Epoch accuracy:95.12%, Validation loss:0.723, Validation accuracy:75.52%\n",
      "Epoch 295: iteration 106/107\n",
      "Epoch 295 --- loss:0.160, Epoch accuracy:94.97%, Validation loss:0.674, Validation accuracy:75.99%\n",
      "Epoch 296: iteration 106/107\n",
      "Epoch 296 --- loss:0.165, Epoch accuracy:94.10%, Validation loss:0.733, Validation accuracy:74.83%\n",
      "Epoch 297: iteration 106/107\n",
      "Epoch 297 --- loss:0.145, Epoch accuracy:95.64%, Validation loss:0.707, Validation accuracy:75.06%\n",
      "Epoch 298: iteration 106/107\n",
      "Epoch 298 --- loss:0.155, Epoch accuracy:94.94%, Validation loss:0.720, Validation accuracy:74.83%\n",
      "Epoch 299: iteration 106/107\n",
      "Epoch 299 --- loss:0.150, Epoch accuracy:95.18%, Validation loss:0.748, Validation accuracy:72.49%\n",
      "Epoch 300: iteration 106/107\n",
      "Epoch 300 --- loss:0.148, Epoch accuracy:94.74%, Validation loss:0.714, Validation accuracy:74.59%\n",
      "Epoch 301: iteration 106/107\n",
      "Epoch 301 --- loss:0.137, Epoch accuracy:95.52%, Validation loss:0.694, Validation accuracy:73.89%\n",
      "Epoch 302: iteration 106/107\n",
      "Epoch 302 --- loss:0.149, Epoch accuracy:94.62%, Validation loss:0.667, Validation accuracy:74.59%\n",
      "Epoch 303: iteration 106/107\n",
      "Epoch 303 --- loss:0.151, Epoch accuracy:94.89%, Validation loss:0.645, Validation accuracy:77.39%\n",
      "Epoch 304: iteration 106/107\n",
      "Epoch 304 --- loss:0.142, Epoch accuracy:95.09%, Validation loss:0.658, Validation accuracy:75.76%\n",
      "Epoch 305: iteration 106/107\n",
      "Epoch 305 --- loss:0.147, Epoch accuracy:95.15%, Validation loss:0.649, Validation accuracy:75.29%\n",
      "Epoch 306: iteration 106/107\n",
      "Epoch 306 --- loss:0.140, Epoch accuracy:95.50%, Validation loss:0.659, Validation accuracy:76.22%\n",
      "Epoch 307: iteration 106/107\n",
      "Epoch 307 --- loss:0.169, Epoch accuracy:93.98%, Validation loss:0.709, Validation accuracy:76.92%\n",
      "Epoch 308: iteration 106/107\n",
      "Epoch 308 --- loss:0.150, Epoch accuracy:95.03%, Validation loss:0.728, Validation accuracy:73.89%\n",
      "Epoch 309: iteration 106/107\n",
      "Epoch 309 --- loss:0.160, Epoch accuracy:94.62%, Validation loss:0.704, Validation accuracy:75.76%\n",
      "Epoch 310: iteration 106/107\n",
      "Epoch 310 --- loss:0.158, Epoch accuracy:94.77%, Validation loss:0.684, Validation accuracy:76.22%\n",
      "Epoch 311: iteration 106/107\n",
      "Epoch 311 --- loss:0.141, Epoch accuracy:95.67%, Validation loss:0.656, Validation accuracy:76.92%\n",
      "Epoch 312: iteration 106/107\n",
      "Epoch 312 --- loss:0.151, Epoch accuracy:95.12%, Validation loss:0.692, Validation accuracy:77.39%\n",
      "Epoch 313: iteration 106/107\n",
      "Epoch 313 --- loss:0.157, Epoch accuracy:94.65%, Validation loss:0.708, Validation accuracy:75.52%\n",
      "Epoch 314: iteration 106/107\n",
      "Epoch 314 --- loss:0.139, Epoch accuracy:95.18%, Validation loss:0.671, Validation accuracy:76.92%\n",
      "Epoch 315: iteration 106/107\n",
      "Epoch 315 --- loss:0.146, Epoch accuracy:95.23%, Validation loss:0.749, Validation accuracy:72.26%\n",
      "Epoch 316: iteration 106/107\n",
      "Epoch 316 --- loss:0.145, Epoch accuracy:95.15%, Validation loss:0.666, Validation accuracy:77.62%\n",
      "Epoch 317: iteration 106/107\n",
      "Epoch 317 --- loss:0.149, Epoch accuracy:94.71%, Validation loss:0.688, Validation accuracy:78.09%\n",
      "Epoch 318: iteration 106/107\n",
      "Epoch 318 --- loss:0.160, Epoch accuracy:94.86%, Validation loss:0.664, Validation accuracy:75.52%\n",
      "Epoch 319: iteration 106/107\n",
      "Epoch 319 --- loss:0.150, Epoch accuracy:94.91%, Validation loss:0.709, Validation accuracy:75.29%\n",
      "Epoch 320: iteration 106/107\n",
      "Epoch 320 --- loss:0.144, Epoch accuracy:95.06%, Validation loss:0.709, Validation accuracy:75.29%\n",
      "Epoch 321: iteration 106/107\n",
      "Epoch 321 --- loss:0.161, Epoch accuracy:94.89%, Validation loss:0.711, Validation accuracy:72.96%\n",
      "Epoch 322: iteration 106/107\n",
      "Epoch 322 --- loss:0.159, Epoch accuracy:94.65%, Validation loss:0.707, Validation accuracy:73.43%\n",
      "Epoch 323: iteration 106/107\n",
      "Epoch 323 --- loss:0.145, Epoch accuracy:95.67%, Validation loss:0.767, Validation accuracy:74.13%\n",
      "Epoch 324: iteration 106/107\n",
      "Epoch 324 --- loss:0.148, Epoch accuracy:94.74%, Validation loss:0.680, Validation accuracy:76.46%\n",
      "Epoch 325: iteration 106/107\n",
      "Epoch 325 --- loss:0.143, Epoch accuracy:95.76%, Validation loss:0.711, Validation accuracy:74.59%\n",
      "Epoch 326: iteration 106/107\n",
      "Epoch 326 --- loss:0.165, Epoch accuracy:94.27%, Validation loss:0.767, Validation accuracy:75.29%\n",
      "Epoch 327: iteration 106/107\n",
      "Epoch 327 --- loss:0.156, Epoch accuracy:94.77%, Validation loss:0.789, Validation accuracy:73.19%\n",
      "Epoch 328: iteration 106/107\n",
      "Epoch 328 --- loss:0.145, Epoch accuracy:95.38%, Validation loss:0.718, Validation accuracy:77.39%\n",
      "Epoch 329: iteration 106/107\n",
      "Epoch 329 --- loss:0.146, Epoch accuracy:95.03%, Validation loss:0.792, Validation accuracy:74.59%\n",
      "Epoch 330: iteration 106/107\n",
      "Epoch 330 --- loss:0.161, Epoch accuracy:95.03%, Validation loss:0.675, Validation accuracy:77.62%\n",
      "Epoch 331: iteration 106/107\n",
      "Epoch 331 --- loss:0.138, Epoch accuracy:95.58%, Validation loss:0.716, Validation accuracy:74.13%\n",
      "Epoch 332: iteration 106/107\n",
      "Epoch 332 --- loss:0.144, Epoch accuracy:95.44%, Validation loss:0.674, Validation accuracy:76.69%\n",
      "Epoch 333: iteration 106/107\n",
      "Epoch 333 --- loss:0.133, Epoch accuracy:95.67%, Validation loss:0.722, Validation accuracy:73.19%\n",
      "Epoch 334: iteration 106/107\n",
      "Epoch 334 --- loss:0.146, Epoch accuracy:95.26%, Validation loss:0.773, Validation accuracy:74.59%\n",
      "Epoch 335: iteration 106/107\n",
      "Epoch 335 --- loss:0.129, Epoch accuracy:95.67%, Validation loss:0.718, Validation accuracy:74.59%\n",
      "Epoch 336: iteration 106/107\n",
      "Epoch 336 --- loss:0.139, Epoch accuracy:95.26%, Validation loss:0.723, Validation accuracy:75.99%\n",
      "Epoch 337: iteration 106/107\n",
      "Epoch 337 --- loss:0.153, Epoch accuracy:94.89%, Validation loss:0.684, Validation accuracy:73.89%\n",
      "Epoch 338: iteration 106/107\n",
      "Epoch 338 --- loss:0.148, Epoch accuracy:95.06%, Validation loss:0.666, Validation accuracy:76.92%\n",
      "Epoch 339: iteration 106/107\n",
      "Epoch 339 --- loss:0.129, Epoch accuracy:95.52%, Validation loss:0.641, Validation accuracy:76.92%\n",
      "Epoch 340: iteration 106/107\n",
      "Epoch 340 --- loss:0.141, Epoch accuracy:95.41%, Validation loss:0.747, Validation accuracy:73.66%\n",
      "Epoch 341: iteration 106/107\n",
      "Epoch 341 --- loss:0.162, Epoch accuracy:94.89%, Validation loss:0.670, Validation accuracy:76.46%\n",
      "Epoch 342: iteration 106/107\n",
      "Epoch 342 --- loss:0.163, Epoch accuracy:94.36%, Validation loss:0.647, Validation accuracy:77.16%\n",
      "Epoch 343: iteration 106/107\n",
      "Epoch 343 --- loss:0.143, Epoch accuracy:95.52%, Validation loss:0.693, Validation accuracy:75.06%\n",
      "Epoch 344: iteration 106/107\n",
      "Epoch 344 --- loss:0.151, Epoch accuracy:94.83%, Validation loss:0.662, Validation accuracy:77.16%\n",
      "Epoch 345: iteration 106/107\n",
      "Epoch 345 --- loss:0.156, Epoch accuracy:95.00%, Validation loss:0.687, Validation accuracy:77.16%\n",
      "Epoch 346: iteration 106/107\n",
      "Epoch 346 --- loss:0.164, Epoch accuracy:94.33%, Validation loss:0.696, Validation accuracy:75.76%\n",
      "Epoch 347: iteration 106/107\n",
      "Epoch 347 --- loss:0.133, Epoch accuracy:95.73%, Validation loss:0.643, Validation accuracy:76.69%\n",
      "Epoch 348: iteration 106/107\n",
      "Epoch 348 --- loss:0.135, Epoch accuracy:95.61%, Validation loss:0.674, Validation accuracy:75.99%\n",
      "Epoch 349: iteration 106/107\n",
      "Epoch 349 --- loss:0.148, Epoch accuracy:95.09%, Validation loss:0.674, Validation accuracy:75.29%\n",
      "Epoch 350: iteration 106/107\n",
      "Epoch 350 --- loss:0.142, Epoch accuracy:94.91%, Validation loss:0.690, Validation accuracy:75.52%\n",
      "Epoch 351: iteration 106/107\n",
      "Epoch 351 --- loss:0.154, Epoch accuracy:94.94%, Validation loss:0.660, Validation accuracy:75.29%\n",
      "Epoch 352: iteration 106/107\n",
      "Epoch 352 --- loss:0.139, Epoch accuracy:95.64%, Validation loss:0.665, Validation accuracy:74.83%\n",
      "Epoch 353: iteration 106/107\n",
      "Epoch 353 --- loss:0.146, Epoch accuracy:95.35%, Validation loss:0.673, Validation accuracy:75.06%\n",
      "Epoch 354: iteration 106/107\n",
      "Epoch 354 --- loss:0.150, Epoch accuracy:94.77%, Validation loss:0.665, Validation accuracy:75.76%\n",
      "Epoch 355: iteration 106/107\n",
      "Epoch 355 --- loss:0.134, Epoch accuracy:95.55%, Validation loss:0.653, Validation accuracy:77.16%\n",
      "Epoch 356: iteration 106/107\n",
      "Epoch 356 --- loss:0.149, Epoch accuracy:94.86%, Validation loss:0.645, Validation accuracy:75.06%\n",
      "Epoch 357: iteration 106/107\n",
      "Epoch 357 --- loss:0.136, Epoch accuracy:95.47%, Validation loss:0.724, Validation accuracy:74.59%\n",
      "Epoch 358: iteration 106/107\n",
      "Epoch 358 --- loss:0.144, Epoch accuracy:95.06%, Validation loss:0.759, Validation accuracy:71.79%\n",
      "Epoch 359: iteration 106/107\n",
      "Epoch 359 --- loss:0.142, Epoch accuracy:95.15%, Validation loss:0.707, Validation accuracy:75.76%\n",
      "Epoch 360: iteration 106/107\n",
      "Epoch 360 --- loss:0.153, Epoch accuracy:94.97%, Validation loss:0.746, Validation accuracy:75.06%\n",
      "Epoch 361: iteration 106/107\n",
      "Epoch 361 --- loss:0.144, Epoch accuracy:95.00%, Validation loss:0.697, Validation accuracy:74.59%\n",
      "Epoch 362: iteration 106/107\n",
      "Epoch 362 --- loss:0.141, Epoch accuracy:95.58%, Validation loss:0.698, Validation accuracy:73.89%\n",
      "Epoch 363: iteration 106/107\n",
      "Epoch 363 --- loss:0.133, Epoch accuracy:95.76%, Validation loss:0.659, Validation accuracy:73.66%\n",
      "Epoch 364: iteration 106/107\n",
      "Epoch 364 --- loss:0.142, Epoch accuracy:95.29%, Validation loss:0.672, Validation accuracy:76.46%\n",
      "Epoch 365: iteration 106/107\n",
      "Epoch 365 --- loss:0.143, Epoch accuracy:95.26%, Validation loss:0.684, Validation accuracy:75.52%\n",
      "Epoch 366: iteration 106/107\n",
      "Epoch 366 --- loss:0.148, Epoch accuracy:95.12%, Validation loss:0.673, Validation accuracy:77.39%\n",
      "Epoch 367: iteration 106/107\n",
      "Epoch 367 --- loss:0.147, Epoch accuracy:94.86%, Validation loss:0.685, Validation accuracy:75.99%\n",
      "Epoch 368: iteration 106/107\n",
      "Epoch 368 --- loss:0.142, Epoch accuracy:95.50%, Validation loss:0.641, Validation accuracy:78.79%\n",
      "Epoch 369: iteration 106/107\n",
      "Epoch 369 --- loss:0.141, Epoch accuracy:95.26%, Validation loss:0.671, Validation accuracy:76.92%\n",
      "Epoch 370: iteration 106/107\n",
      "Epoch 370 --- loss:0.132, Epoch accuracy:95.70%, Validation loss:0.669, Validation accuracy:76.69%\n",
      "Epoch 371: iteration 106/107\n",
      "Epoch 371 --- loss:0.141, Epoch accuracy:95.26%, Validation loss:0.666, Validation accuracy:76.22%\n",
      "Epoch 372: iteration 106/107\n",
      "Epoch 372 --- loss:0.141, Epoch accuracy:95.50%, Validation loss:0.652, Validation accuracy:77.86%\n",
      "Epoch 373: iteration 106/107\n",
      "Epoch 373 --- loss:0.131, Epoch accuracy:95.41%, Validation loss:0.646, Validation accuracy:76.92%\n",
      "Epoch 374: iteration 106/107\n",
      "Epoch 374 --- loss:0.144, Epoch accuracy:95.03%, Validation loss:0.688, Validation accuracy:77.62%\n",
      "Epoch 375: iteration 106/107\n",
      "Epoch 375 --- loss:0.144, Epoch accuracy:95.52%, Validation loss:0.665, Validation accuracy:78.32%\n",
      "Epoch 376: iteration 106/107\n",
      "Epoch 376 --- loss:0.156, Epoch accuracy:94.89%, Validation loss:0.707, Validation accuracy:74.83%\n",
      "Epoch 377: iteration 106/107\n",
      "Epoch 377 --- loss:0.159, Epoch accuracy:94.13%, Validation loss:0.743, Validation accuracy:72.49%\n",
      "Epoch 378: iteration 106/107\n",
      "Epoch 378 --- loss:0.154, Epoch accuracy:95.18%, Validation loss:0.716, Validation accuracy:75.52%\n",
      "Epoch 379: iteration 106/107\n",
      "Epoch 379 --- loss:0.141, Epoch accuracy:95.44%, Validation loss:0.690, Validation accuracy:73.89%\n",
      "Epoch 380: iteration 106/107\n",
      "Epoch 380 --- loss:0.143, Epoch accuracy:95.20%, Validation loss:0.677, Validation accuracy:75.52%\n",
      "Epoch 381: iteration 106/107\n",
      "Epoch 381 --- loss:0.140, Epoch accuracy:95.32%, Validation loss:0.700, Validation accuracy:73.43%\n",
      "Epoch 382: iteration 106/107\n",
      "Epoch 382 --- loss:0.127, Epoch accuracy:96.08%, Validation loss:0.660, Validation accuracy:76.22%\n",
      "Epoch 383: iteration 106/107\n",
      "Epoch 383 --- loss:0.137, Epoch accuracy:95.15%, Validation loss:0.686, Validation accuracy:75.06%\n",
      "Epoch 384: iteration 106/107\n",
      "Epoch 384 --- loss:0.140, Epoch accuracy:95.12%, Validation loss:0.697, Validation accuracy:74.59%\n",
      "Epoch 385: iteration 106/107\n",
      "Epoch 385 --- loss:0.145, Epoch accuracy:95.32%, Validation loss:0.705, Validation accuracy:75.29%\n",
      "Epoch 386: iteration 106/107\n",
      "Epoch 386 --- loss:0.146, Epoch accuracy:94.77%, Validation loss:0.692, Validation accuracy:75.76%\n",
      "Epoch 387: iteration 106/107\n",
      "Epoch 387 --- loss:0.133, Epoch accuracy:95.61%, Validation loss:0.721, Validation accuracy:75.76%\n",
      "Epoch 388: iteration 106/107\n",
      "Epoch 388 --- loss:0.147, Epoch accuracy:95.12%, Validation loss:0.707, Validation accuracy:75.29%\n",
      "Epoch 389: iteration 106/107\n",
      "Epoch 389 --- loss:0.139, Epoch accuracy:95.18%, Validation loss:0.691, Validation accuracy:73.89%\n",
      "Epoch 390: iteration 106/107\n",
      "Epoch 390 --- loss:0.148, Epoch accuracy:94.89%, Validation loss:0.730, Validation accuracy:73.19%\n",
      "Epoch 391: iteration 106/107\n",
      "Epoch 391 --- loss:0.130, Epoch accuracy:95.96%, Validation loss:0.660, Validation accuracy:76.46%\n",
      "Epoch 392: iteration 106/107\n",
      "Epoch 392 --- loss:0.138, Epoch accuracy:95.35%, Validation loss:0.694, Validation accuracy:74.36%\n",
      "Epoch 393: iteration 106/107\n",
      "Epoch 393 --- loss:0.143, Epoch accuracy:95.12%, Validation loss:0.695, Validation accuracy:74.36%\n",
      "Epoch 394: iteration 106/107\n",
      "Epoch 394 --- loss:0.147, Epoch accuracy:95.15%, Validation loss:0.731, Validation accuracy:74.13%\n",
      "Epoch 395: iteration 106/107\n",
      "Epoch 395 --- loss:0.136, Epoch accuracy:95.58%, Validation loss:0.663, Validation accuracy:76.46%\n",
      "Epoch 396: iteration 106/107\n",
      "Epoch 396 --- loss:0.134, Epoch accuracy:95.67%, Validation loss:0.671, Validation accuracy:75.52%\n",
      "Epoch 397: iteration 106/107\n",
      "Epoch 397 --- loss:0.151, Epoch accuracy:94.91%, Validation loss:0.647, Validation accuracy:77.62%\n",
      "Epoch 398: iteration 106/107\n",
      "Epoch 398 --- loss:0.147, Epoch accuracy:95.41%, Validation loss:0.725, Validation accuracy:76.69%\n",
      "Epoch 399: iteration 106/107\n",
      "Epoch 399 --- loss:0.126, Epoch accuracy:95.90%, Validation loss:0.655, Validation accuracy:77.39%\n",
      "Epoch 400: iteration 106/107\n",
      "Epoch 400 --- loss:0.143, Epoch accuracy:95.03%, Validation loss:0.694, Validation accuracy:73.89%\n",
      "Epoch 401: iteration 106/107\n",
      "Epoch 401 --- loss:0.140, Epoch accuracy:95.67%, Validation loss:0.707, Validation accuracy:75.76%\n",
      "Epoch 402: iteration 106/107\n",
      "Epoch 402 --- loss:0.141, Epoch accuracy:95.52%, Validation loss:0.724, Validation accuracy:75.29%\n",
      "Epoch 403: iteration 106/107\n",
      "Epoch 403 --- loss:0.141, Epoch accuracy:95.23%, Validation loss:0.734, Validation accuracy:75.52%\n",
      "Epoch 404: iteration 106/107\n",
      "Epoch 404 --- loss:0.140, Epoch accuracy:95.41%, Validation loss:0.643, Validation accuracy:77.16%\n",
      "Epoch 405: iteration 106/107\n",
      "Epoch 405 --- loss:0.136, Epoch accuracy:95.47%, Validation loss:0.696, Validation accuracy:75.76%\n",
      "Epoch 406: iteration 106/107\n",
      "Epoch 406 --- loss:0.145, Epoch accuracy:95.41%, Validation loss:0.718, Validation accuracy:73.19%\n",
      "Epoch 407: iteration 106/107\n",
      "Epoch 407 --- loss:0.138, Epoch accuracy:95.58%, Validation loss:0.645, Validation accuracy:78.09%\n",
      "Epoch 408: iteration 106/107\n",
      "Epoch 408 --- loss:0.135, Epoch accuracy:95.50%, Validation loss:0.717, Validation accuracy:75.52%\n",
      "Epoch 409: iteration 106/107\n",
      "Epoch 409 --- loss:0.150, Epoch accuracy:95.12%, Validation loss:0.700, Validation accuracy:75.06%\n",
      "Epoch 410: iteration 106/107\n",
      "Epoch 410 --- loss:0.148, Epoch accuracy:95.23%, Validation loss:0.693, Validation accuracy:76.22%\n",
      "Epoch 411: iteration 106/107\n",
      "Epoch 411 --- loss:0.149, Epoch accuracy:95.20%, Validation loss:0.691, Validation accuracy:74.36%\n",
      "Epoch 412: iteration 106/107\n",
      "Epoch 412 --- loss:0.142, Epoch accuracy:95.32%, Validation loss:0.664, Validation accuracy:75.99%\n",
      "Epoch 413: iteration 106/107\n",
      "Epoch 413 --- loss:0.135, Epoch accuracy:95.73%, Validation loss:0.631, Validation accuracy:77.62%\n",
      "Epoch 414: iteration 106/107\n",
      "Epoch 414 --- loss:0.137, Epoch accuracy:95.26%, Validation loss:0.700, Validation accuracy:76.22%\n",
      "Epoch 415: iteration 106/107\n",
      "Epoch 415 --- loss:0.136, Epoch accuracy:95.58%, Validation loss:0.622, Validation accuracy:76.69%\n",
      "Epoch 416: iteration 106/107\n",
      "Epoch 416 --- loss:0.141, Epoch accuracy:95.23%, Validation loss:0.706, Validation accuracy:75.99%\n",
      "Epoch 417: iteration 106/107\n",
      "Epoch 417 --- loss:0.127, Epoch accuracy:96.08%, Validation loss:0.708, Validation accuracy:77.16%\n",
      "Epoch 418: iteration 106/107\n",
      "Epoch 418 --- loss:0.133, Epoch accuracy:95.70%, Validation loss:0.703, Validation accuracy:74.59%\n",
      "Epoch 419: iteration 106/107\n",
      "Epoch 419 --- loss:0.129, Epoch accuracy:95.67%, Validation loss:0.664, Validation accuracy:78.09%\n",
      "Epoch 420: iteration 106/107\n",
      "Epoch 420 --- loss:0.135, Epoch accuracy:95.52%, Validation loss:0.660, Validation accuracy:76.92%\n",
      "Epoch 421: iteration 106/107\n",
      "Epoch 421 --- loss:0.140, Epoch accuracy:95.15%, Validation loss:0.818, Validation accuracy:73.89%\n",
      "Epoch 422: iteration 106/107\n",
      "Epoch 422 --- loss:0.149, Epoch accuracy:95.00%, Validation loss:0.682, Validation accuracy:76.69%\n",
      "Epoch 423: iteration 106/107\n",
      "Epoch 423 --- loss:0.154, Epoch accuracy:95.20%, Validation loss:0.701, Validation accuracy:76.69%\n",
      "Epoch 424: iteration 106/107\n",
      "Epoch 424 --- loss:0.144, Epoch accuracy:95.47%, Validation loss:0.707, Validation accuracy:77.39%\n",
      "Epoch 425: iteration 106/107\n",
      "Epoch 425 --- loss:0.151, Epoch accuracy:94.91%, Validation loss:0.674, Validation accuracy:76.69%\n",
      "Epoch 426: iteration 106/107\n",
      "Epoch 426 --- loss:0.129, Epoch accuracy:95.58%, Validation loss:0.693, Validation accuracy:76.46%\n",
      "Epoch 427: iteration 106/107\n",
      "Epoch 427 --- loss:0.134, Epoch accuracy:95.52%, Validation loss:0.672, Validation accuracy:76.46%\n",
      "Epoch 428: iteration 106/107\n",
      "Epoch 428 --- loss:0.127, Epoch accuracy:95.84%, Validation loss:0.660, Validation accuracy:75.29%\n",
      "Epoch 429: iteration 106/107\n",
      "Epoch 429 --- loss:0.146, Epoch accuracy:94.94%, Validation loss:0.661, Validation accuracy:79.72%\n",
      "Epoch 430: iteration 106/107\n",
      "Epoch 430 --- loss:0.143, Epoch accuracy:95.52%, Validation loss:0.677, Validation accuracy:77.16%\n",
      "Epoch 431: iteration 106/107\n",
      "Epoch 431 --- loss:0.153, Epoch accuracy:95.47%, Validation loss:0.705, Validation accuracy:75.29%\n",
      "Epoch 432: iteration 106/107\n",
      "Epoch 432 --- loss:0.123, Epoch accuracy:96.05%, Validation loss:0.686, Validation accuracy:78.55%\n",
      "Epoch 433: iteration 106/107\n",
      "Epoch 433 --- loss:0.152, Epoch accuracy:95.03%, Validation loss:0.690, Validation accuracy:76.69%\n",
      "Epoch 434: iteration 106/107\n",
      "Epoch 434 --- loss:0.133, Epoch accuracy:95.61%, Validation loss:0.648, Validation accuracy:79.95%\n",
      "Epoch 435: iteration 106/107\n",
      "Epoch 435 --- loss:0.137, Epoch accuracy:96.02%, Validation loss:0.719, Validation accuracy:75.99%\n",
      "Epoch 436: iteration 106/107\n",
      "Epoch 436 --- loss:0.150, Epoch accuracy:95.00%, Validation loss:0.727, Validation accuracy:76.69%\n",
      "Epoch 437: iteration 106/107\n",
      "Epoch 437 --- loss:0.136, Epoch accuracy:95.64%, Validation loss:0.685, Validation accuracy:77.39%\n",
      "Epoch 438: iteration 106/107\n",
      "Epoch 438 --- loss:0.150, Epoch accuracy:94.51%, Validation loss:0.709, Validation accuracy:75.52%\n",
      "Epoch 439: iteration 106/107\n",
      "Epoch 439 --- loss:0.151, Epoch accuracy:95.03%, Validation loss:0.660, Validation accuracy:78.79%\n",
      "Epoch 440: iteration 106/107\n",
      "Epoch 440 --- loss:0.139, Epoch accuracy:95.44%, Validation loss:0.679, Validation accuracy:79.02%\n",
      "Epoch 441: iteration 106/107\n",
      "Epoch 441 --- loss:0.142, Epoch accuracy:95.52%, Validation loss:0.709, Validation accuracy:76.69%\n",
      "Epoch 442: iteration 106/107\n",
      "Epoch 442 --- loss:0.120, Epoch accuracy:96.16%, Validation loss:0.754, Validation accuracy:76.46%\n",
      "Epoch 443: iteration 106/107\n",
      "Epoch 443 --- loss:0.144, Epoch accuracy:95.32%, Validation loss:0.721, Validation accuracy:73.43%\n",
      "Epoch 444: iteration 106/107\n",
      "Epoch 444 --- loss:0.148, Epoch accuracy:94.91%, Validation loss:0.676, Validation accuracy:75.99%\n",
      "Epoch 445: iteration 106/107\n",
      "Epoch 445 --- loss:0.146, Epoch accuracy:95.15%, Validation loss:0.706, Validation accuracy:76.69%\n",
      "Epoch 446: iteration 106/107\n",
      "Epoch 446 --- loss:0.149, Epoch accuracy:95.20%, Validation loss:0.633, Validation accuracy:78.55%\n",
      "Epoch 447: iteration 48/107"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-23-d14eb4f1b628>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;31m# train it!\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mtrain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moptimizer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_epochs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mX_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mX_valid\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_valid\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-22-3b155e61bacc>\u001b[0m in \u001b[0;36mtrain\u001b[0;34m(optimizer, model, num_epochs, X_train, Y_train, X_valid, Y_valid)\u001b[0m\n\u001b[1;32m     72\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     73\u001b[0m             \u001b[0;31m# Pass input tensors thru 1 training step (fwd+backwards pass)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 74\u001b[0;31m             \u001b[0mloss\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0macc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_step\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_tensor\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mY_tensor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     75\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     76\u001b[0m             \u001b[0;31m# aggregate batch accuracy to measure progress of entire epoch\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-19-72eaed1e66fd>\u001b[0m in \u001b[0;36mtrain_step\u001b[0;34m(X, Y)\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m         \u001b[0;31m# forward pass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m         \u001b[0moutput_logits\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput_softmax\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      9\u001b[0m         \u001b[0mpredictions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput_softmax\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mdim\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m         \u001b[0maccuracy\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mY\u001b[0m\u001b[0;34m==\u001b[0m\u001b[0mpredictions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mY\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    720\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    721\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 722\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    723\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    724\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-15-3e7c192fbfc0>\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, x)\u001b[0m\n\u001b[1;32m    130\u001b[0m         \u001b[0;31m# create final feature embedding from 1st convolutional layer\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    131\u001b[0m         \u001b[0;31m# input features pased through 4 sequential 2D convolutional layers\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 132\u001b[0;31m         \u001b[0mconv2d_embedding1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconv2Dblock1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# x == N/batch * channel * freq * time\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    133\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    134\u001b[0m         \u001b[0;31m# flatten final 64*1*4 feature map from convolutional layers to length 256 1D array\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    720\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    721\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 722\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    723\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    724\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m    115\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    116\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 117\u001b[0;31m             \u001b[0minput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodule\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    118\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    119\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    720\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    721\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 722\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    723\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    724\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/batchnorm.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m    134\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrunning_mean\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtraining\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrack_running_stats\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    135\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrunning_var\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtraining\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrack_running_stats\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 136\u001b[0;31m             self.weight, self.bias, bn_training, exponential_average_factor, self.eps)\n\u001b[0m\u001b[1;32m    137\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    138\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m__getattr__\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m    756\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_load_state_dict_pre_hooks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mOrderedDict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    757\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 758\u001b[0;31m     \u001b[0;32mdef\u001b[0m \u001b[0m__getattr__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mUnion\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mTensor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'Module'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    759\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0;34m'_parameters'\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__dict__\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    760\u001b[0m             \u001b[0m_parameters\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__dict__\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'_parameters'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "# choose number of epochs higher than reasonable so we can manually stop training \n",
    "num_epochs = 500\n",
    "\n",
    "# train it!\n",
    "train(optimizer, model, num_epochs, X_train, y_train, X_valid, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cPDpzMtU60D9"
   },
   "source": [
    "# Check the Loss Curve's Behaviour\n",
    "Let's see if we missed something egregious during training. Note that validation sets achieved 78-82% accuracy depending on the random split of data (checked after tuning; random seed was specified before splitting data while tuning the model)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 318
    },
    "id": "m4gI1cj460D9",
    "outputId": "18246956-a5b9-4742-c5db-c8ae0488f0bc"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAEtCAYAAAAoQkgIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd1wT9xsH8E8GG2RvRa0aRJGNVkRBFHErFrXOOqp1VFvt0g6rtv4c1Vq1bsWBo9aJq25BEBUQBBeKG0EElL1Ckvv9QZMSEyCsJMjzfr36Kt59c/fkktxz33XHYhiGASGEEELUFlvVARBCCCGkapSsCSGEEDVHyZoQQghRc5SsCSGEEDVHyZoQQghRc5SsCSGEEDWnULL28/ODvb09bty40dDxqExiYiIWLFiAfv36wd3dHY6OjvD29sbkyZOxd+9eFBQUqDpEpbt27RrGjh0LNzc32Nvbw97eHi9fvlRZPOIYKv7n6uqKwYMHY/Xq1cjNzVVZbNWZN28e7O3tceTIEYWW18aNGzdgb2+PcePGKfyaI0eOwN7eHvPmzavz/hUhjtHe3h7Lly+vsuy6detgb2+PdevWKbRcnqNHj8Le3h5+fn4oLCystNzr16/h4eGB9u3bq/w8N2fOHNjb2+PPP/+Uuz41NVVyDP/44w+5ZV6/fg17e3t06NAB+fn5DRluvav4+z5+/Hil5QQCAby9vSVl9+zZ0+Cx1ffvRRy7Ipp8zZrP52P+/PkYPnw4Dhw4gNLSUnTu3BkBAQFo3bo1YmJisHjxYvTu3RupqamqDldpXr16hRkzZiA2NhaOjo4YOnQoAgMDoaurq+rQEBAQgMDAQAQGBqJTp054/vw5Nm3ahMDAQLx+/VrV4ZEqVLwoOXHiBAQCQYPuLzAwEL6+vkhNTcVvv/1WabkFCxYgPz8fY8aMQZcuXRo0pup4enoCAGJiYuSur7g8NjZWbpno6GgAQPv27WFgYFDPEVatJhdT1QkNDa10XWRkJDIzM+u8j8aCq+oAVIlhGMyaNQthYWGwtbXFr7/+Ci8vL6kyRUVFOHDgANavX4+8vDzY2tqqKFrlunr1KoqKijBkyBCsWLFC1eFI+fbbb9G8eXPJv58/f44JEyYgNTUVy5cvx++//67C6BoXf39/ODs7K+WEXlhYiHPnzoHFYsHMzAyZmZmIiIhAz549G3S/ixcvxsCBA/HXX3+hX79+Msn42LFjCAsLQ4sWLfDVV181aCyK6Ny5MwDg1q1b4PP50NTUlFovTsQODg5ISEhAaWkptLS0pMqIE7p4W41Rx44dERUVhdevX8PS0lJmvfjCz9HREXfu3FF2eErXpGvW+/btQ1hYGIyNjbF//36ZRA0Aurq6mDhxIo4cOQJTU1MVRKka6enpAIAWLVqoOJLqtWzZErNnzwYAXLp0qcFra+8TAwMDtGnTBhYWFg2+rzNnzqCoqAgeHh4YO3YsgPJm6oZmaWmJ77//HgzD4IcffkBRUZFkXWZmJpYuXQoWi4UlS5aoRctR27ZtYWJigpKSEty+fVtmfUxMDFq0aIH+/fuDz+cjISFBpow4oYtr6Y3RsGHDIBKJ5Nauc3NzcfnyZXTs2BHt2rVTQXTK12DJ+sKFC5g0aRI6d+4MR0dH+Pn5YcGCBZU2Jd+/fx9fffUV/P394eTkBE9PTwQEBGDevHm4e/euVNmMjAwsX74cAwYMgJubG1xdXdGzZ09Mnz4dZ86cUSg+kUiEbdu2AQBmzZol98qtIjs7O6kT2rhx46rsxxf387/bx1tx+enTpzFq1Ci4u7vD3t4e2dnZcHR0hIuLS6V95Hw+H56ennBwcJBp8n379i1WrlyJAQMGwNnZGa6urhgxYgQOHjwIRe8qK+6TETdh/fnnn5J+lXf7aWJjYzFjxgx07doVjo6O6N69O77++ms8fPhQ7rbF22EYBvv27cOwYcPg6uoKDw8PhWKrSocOHQAAxcXFePv2LQDg7NmzmDdvHvr37w93d3c4OTmhb9++WL58uaTMu6r6fPLy8gAACQkJWLZsGYYNGyZ57z4+PlW+99oKCwvD1KlTpfYzf/58pKSk1Mv2q+qDu3DhAiZOnIgePXrA0dERXbt2xZAhQ7B06dJKj19VxIl56NChGDp0KNhsNi5duoScnJw6v4/qBAYGwsfHBykpKVi1apVk+aJFi5CTk4PRo0dL1bhrcv6qbqxAbfo5xUn23Wbu169f48WLF/D09JT8bt5tLs/KysLTp0/BZrMlZd68eYOdO3di0qRJ8PPzQ6dOneDp6YkxY8bg2LFjcmOo+L74fD7Wrl0Lf39/ODo6okePHliyZInUhQ9Q/vsR97VXPHfUplk8ICAAurq6cpP1yZMnwefzERgYWOU2RCIRjhw5gtGjR8PDwwOdOnVCQEBAlecAADh9+jSGDx8OZ2dndOnSBdOmTZPJQ/I8fPgQ8+bNg6+vLxwdHdGlSxdMnTq1XsZBNEiyXr58OWbOnInr16+jffv26NOnDzQ0NHDgwAEMHTpU5kowMjISQUFBOHnyJAwMDODn5wcPDw/o6OggNDQUV69elZTNyMhAYGAggoODUVJSgq5du6JHjx6wtLTEtWvXcPDgQYViTEpKQlpaGlgsFgYOHFiv718R27Ztw5w5c8AwDHx9fdGxY0dwuVz4+vqiuLgYZ8+elfu6ixcvIi8vD15eXlIXGElJSRg8eDC2bt2K4uJidOvWDa6urnj06BF+/PFHfP311wrFZWdnh8DAQLRv3x5AeZ+XuH/Y3d1dUi4kJARjx47FxYsX0bJlSwQEBMDExAQnTpzARx99hIsXL1a6j0WLFmHJkiXQ19dHz5490bZtW4Viq0rFixtxs+GcOXNw7tw56Ovrw9vbG126dEFeXh6Cg4MRFBRU5Y9V3ufDYrEAAKtXr0ZISAiEQiFcXFzg6+sLbW1tnDhxAsOHD6+0r7GmFi5ciM8++wxRUVFo1aoV/Pz8oK+vjyNHjiAwMBCJiYn1sh951qxZg5kzZyImJgatWrVCQEAAOnTogOLiYuzcuRMvXryo0fZSUlIQGxsLHR0d9O3bF1ZWVujatSvKyspw8uTJBnoX0n755Rc0a9YMe/fuRUxMDE6fPo3z58+jefPmUr+Pmp6/GoK4+VpcQxYTf7fEg2C1tbVlvm/if9vb28PQ0BBA+Tl26dKlePr0Kezs7ODv7w8ej4f4+Hh89913WLRoUaWxlJWVYfLkyQgJCUG7du3g5eWFgoIC7N69G7NmzZIqGxAQIPfcERgYCAcHhxodA11dXfTp0wePHj2SaWE4duwYNDQ0MGDAgEpfzzAM5s6di/nz5+P27dtwcXFBr169UFxcjODgYAwbNgzPnz+Xed3GjRsxZ84c3LlzB05OTujWrRuePHmCjz/+uMrPPjQ0FMOGDcPRo0dhaGgIPz8/tG7dGpGRkfjkk0+wf//+Gr1/eW+oWj179mR4PB5z/fr1asteunSJ4fF4jLu7O5OQkCBZLhQKmeXLlzM8Ho/x9fVlSktLJevGjh3L8Hg85tSpUzLbS09PZ5KTkyX/XrduHcPj8Ziff/5ZpmxBQQETFxenyFti/v77b4bH4zG9e/dWqPy7xDFXdkzExywlJUXu8o4dOzIREREyrzt//jzD4/GYsWPHyt3uZ599xvB4PCY0NFSyrKioSLLdHTt2MEKhULIuPT2dCQwMZHg8HnPw4EGF39/atWsZHo/HrF27VmbdvXv3GAcHB6Zjx47MpUuXpNaFhIQwPB6PcXNzYzIzM6XW8Xg8hsfjMZ6enszdu3cVjuXd1797TBmGYVasWMHweDzGx8dHsuz06dNMcXGxVLmSkhLm+++/Z3g8HvPTTz/JbKe6z4dhGCY8PJzJysqSWS7+TvXt25cRiURS67777juGx+Mxhw8fVmj5nj17GB6PxwwZMoR59uyZ1Lp9+/YxPB6P6dWrF1NWViZZfv369Sq/O/IcPnyY4fF4zHfffSdZVlJSwnTq1IlxcXGR2TfDMMz9+/flvv+q/PHHHwyPx2O+/vprybLjx48zPB6PCQwMlPuayr6DVX03qyN+v71792Y+/PBDxt7eXuo3XJvzV3XHXd4xrk5SUhLD4/EYFxcXRiAQSJb/9NNPDI/Hk3wuY8eOZZydnRk+ny8ps2jRIobH4zG//vqrZNmjR4+k3o/Y8+fPGV9fX4bH4zHx8fFS68Tvi8fjMSNHjmRycnKkXufu7s7weDwmOjpa6nV1+XwY5r/feUFBARMVFcXweDxm8eLFUu+Fx+MxM2fOZBjmv99QSEiI1HbE5yIfHx+p73FpaSkzd+5chsfjMUFBQVKvuXPnDtO+fXvG0dGRiYqKkiwXCoXML7/8Iont3c/y3r17TMeOHRl3d3ep1zEMw8THxzMeHh5Mx44dmcePH8t9r4qo95r1jh07AACTJ0+Gk5OTZDmbzcbcuXNhZ2eHtLQ0/PPPP5J14hqOt7e3zPYsLS2lal5v3ryptKyenh5cXV0VijM7OxsAYGJiolD5+jZs2DC578HHxwfGxsaIiYlBWlqa1Lq3b98iIiICenp68Pf3lyw/cuQIUlNTMXToUEyYMAFs9n8fq6WlJX755RcAwN69e+sl9t27d0MoFGLo0KEyg4PGjh2Lzp07o6CgAH///bfc13/66aeSZuu6Sk9Px5YtW7Br1y4AwJgxYyTr+vXrB21tbanyWlpa+Omnn8DlcnHu3LlKt1vZ5wMAPXr0kDt+Yfjw4XB1dcWTJ0/w6NGj2rwdAIBQKMTGjRvBZrOxZs0atGzZUmr9qFGj0LNnT6SkpCA8PLzW+6lMYWEhSktLYWdnJ7NvoLzGVJPxGwzDSJoyKzZb+vv7Q19fH3fv3q337oPKDBs2DD4+Pnjx4gXevn0r0/xdm/NXQ+DxeDAyMkJRUZFU82tsbCwsLCwkn4uHhweKi4ulBljJG1zWpk0bqfcjZmdnh+nTpwNApa15bDYbv/76q6SWLn7d4MGDAaBBp7p9+OGHsLW1lTR7A9LdKVURf5Zz586V+h5ramri559/hoGBARITE6W6Gvbs2QORSISPPvoIXbt2lSxns9n49ttvKx3bsWnTJpSVlWH+/PlSrwMAFxcXzJgxA2VlZThw4EAN3r20ek3WAoEA8fHxACC3L4HL5WLIkCEApPtZHB0dAQDffPMN4uPjIRQKK92HuOyqVatw8eJFFBcX11v8ytS7d2+5yzU0NDBw4ECpE5zYyZMnIRAI0LdvX+jo6EiWR0REAAD69u0rd5sdOnSArq4ukpKSUFpaWufYxV/uyn4sH330EYDKp5706tWrTvvv1auXpB/Mx8cHq1atQllZGUaPHo3JkydLlX38+DF27tyJX375BfPnz8e8efOwcOFCaGhoIDs7u9K52ZV9PmJv3rzBwYMHsWzZMvzwww+YN28e5s2bh6ysLADAs2fPav3+7t+/j8zMTDg4OMhNlsB/fZoN0SRrYmICGxsbJCUlYcWKFXj69Gmdtnf9+nWkpqbC2toaH374oWS5trY2+vXrB0A5A83EKjZ5V/y7tuevhsBisWSmcL158waPHz+WGuPxbpns7GwkJyeDxWLJjAUpKytDeHg41q5diwULFkh+D+IkXdl31traWm5XVevWrQGUd002FBaLhcGDByMnJwfh4eEQCoUIDQ2FsbExfHx8Kn1deno6Xr58WWlTebNmzSQVnoqfpfjvQYMGybxGU1NT7jlWJBIhMjISHA4Hffr0kRuP+LOoy++1Xqdu5eTkgM/nQ0NDo9IBW+LRxRUHR3311Vd48uQJwsLCEBYWBl1dXTg5OcHLywtDhw6V2lZgYCCio6Nx7NgxzJgxA1wuF/b29ujSpQsGDx6scL+IsbExANRqoEx9qGoK2NChQxESEoLQ0FDJVS8AyUCQd5OkeLDRtGnTqt1vTk5OtYPpqiP+7CobKS6eVlXZnOe6Tn8TDzxhsVjQ1tZGixYt4OPjgzZt2kjKCAQCLFiwAIcPH65yWwUFBVI1BkVi3Lt3L5YvX17lhU9dbqIj/jzv3r1b7Q0TGur7+9tvv2HOnDnYvn07tm/fDjMzM7i6usLHxwcDBw6UulisjniKzZAhQ6RafYDy3/PBgwdx4sQJfPXVV+ByG342acUR3xX/ru35q6F4enri/PnziImJweTJkyWJpGISdnFxAZfLRXR0NKZOnYqbN2+CYRjweDzJOQ4ov2idMWNGlReRlX1nra2t5S7X09MDAEmNt6EEBgZi48aNOHbsGLS1tZGRkYFx48ZBQ0Oj0teIPx8bGxtwOBy5ZeR9luK/K04NrUjeeSEnJ0dy7KobLFuX36tazLO2sLDAgQMHcPPmTVy5cgUxMTG4efMmrl+/jg0bNmDNmjXw9fUFUN4csXz5ckyZMgWXL1/GjRs3EB8fj7t37yI4OBgzZszAF198Ue0+O3bsCKD8xJibmyv3hF0XIpGoyvXvzousyNHREe3atUNycjISEhLg7OyMx48f4+7du7C1tZWZjiFuifDz86v2fVT1BVeWd5uma+rdedby7Nq1C4cPH4alpSXmz58PV1dXmJiYSAafeXt7IzMzs9JR8pV9PomJifjll1/A5XIxf/58+Pr6wsrKSvKevvrqK5w8eVLh0ffyiL87NjY21d6gw9nZudb7qYqHhwfOnTuHyMhIRERE4ObNmzh//jzOnz+P9evXY+/evQpddBUUFOD8+fMAykdYvztgSkxZc65VobpzQWXEzdg3b96ESCSStGhV/P3r6urCwcFB0iJZ2fzqL774As+ePUPv3r0xZcoUtG7dGvr6+uBwOIiMjJRpkaro3QssZWvZsiXc3NwQHh6OkpISAPJbPlRFfP4Vt4pWpeIFVE3Va7I2MjKCpqYm+Hw+0tPT5V6RiWsN7165stlseHp6Sr6IBQUF2Lx5M7Zs2YIff/wRkZGRUuXbtm2Ltm3bYsqUKRAIBJJpOhs3bsTAgQOlalnytG/fHtbW1nj16hVOnjwp1depCHHSe3fqAlBeq6vrnXWGDBmClStX4tixY3B2dpaqVYtHJYtZW1vj6dOnGD9+vEx/SUOwtLTEixcvkJKSIrcGIp6uVtcafF2Im/YWLVokkwCKiookzdU1de7cOTAMg3HjxmHChAky6+WNLq0pKysrAOWf67Jly+q8vdrS0dGBv7+/pLkwNTUVP//8MyIiIrBq1SqFbj7zzz//SLqqquvHP3r0qEqTdW3PX+JzQWW3MxXfs6CmxKO5c3Nz8eDBA0RHR8PIyEhmXrGnpydu376Ne/fuSZJ1xYT++PFjJCcnw8zMDGvXrpWpadbHd7ahDRs2DHFxcYiMjASPx5NUtioj/nzS0tIgFArl1q7lfZaWlpZISUlBamqq3POXvKl7xsbG0NLSgkAgwOLFi2VuYlNf6vWSicvlSgZ4yZu7JxQKJfd6rW6yvr6+PubOnQsNDQ1kZmZW2XzA5XIxYMAAeHp6gmEYhQarsNlsydXkunXrqm3WSklJkeqbEQ80kNefd+PGjTrfmGPw4MFgs9k4ffo0SktLceLECQDy+4m7d+8OAArPMa8rcVNPZfMzxU3Pqrwhg7gvWt4J99SpU7Wu+Yq3K06oFT1+/Bj379+v1XYrcnJygpGRERITE/Hq1as6b6++2NraYsaMGQCABw8eKPQacV/0Dz/8gAcPHsj9Lzw8HCwWS2lzritT2/OX+Fzw8uVLlJWVybzu3YqGothstmS65IULF/Dw4UO4u7vLXKyLy1y+fFny/atYsxZ/Z83NzeUmrdOnT9cqvsqIL17q8+ZE/fr1g4WFBYyMjDB8+PBqy1tZWaF58+YoKyvDqVOnZNbn5+fjwoULAKQ/S/Hf8qYT8vl8uYPwuFwuvLy8IBQKJdtsCPXeviGubWzfvl1qbpxIJMIff/yB58+fw8bGRjKwBACCg4PlXn1evXoVZWVl0NfXl9wO8dixY7h3755M2fT0dCQlJQEobz5UxJgxY9C9e3dkZ2dj1KhRuHbtmkyZkpIShISEYNiwYZKR6AAkzZP79u2TWv78+XPJ6Ou6sLS0hJeXF3JycrBixQq8evUKbm5usLOzkyk7YsQIWFlZ4cCBA9iyZYvcPqRHjx5VOfq5JsaPHw8Oh4Njx47JjEbet28foqOjoa+vr9CPqqGIB7/s27dPKjHfv3+/Trcj/eCDDwCUz6msWJN6+/Ytvv/++3o5QWloaGD69OkoKyvDjBkz5F4AFBcX48SJE7VuIahKamoqDh48KLcPMywsDEDl/ZgVPX/+HDdv3pRcTFfGysoKnp6eSp1zXZnanL+aN28OW1tb5ObmIiQkRLKcYRhs3LhRMmitNsRJNyQkBAzDyL0AFidw8Uhm8R3QxFq1agU2m43k5GSpkc8Mw2DTpk2V3l+8tsQ10idPntTbNvX19REREYEbN25g/PjxCr1G/Fn+/vvvUvcF4PP5WLx4MfLy8uDk5CTVzzx69Giw2WwcOnRIapS7SCTCqlWrKq3UicdPLVq0SG7CFgqFuH79Om7duqVQ7PLUqBl80aJF0NfXl7tOT08PO3bsgJ+fHyZNmoTg4GCMHDkSnp6eMDMzw507d/Ds2TM0a9YMq1evlmoq2LBhA1asWIG2bdvigw8+gIaGBl6+fCkZOSeuYQPlzZDfffcdrK2t0b59e+jr6+Pt27eIjY1FaWkp+vXrp3A/HpvNxvr16/Hjjz/i+PHjmDBhAmxtbdG+fXvo6OggMzMTCQkJKCkpgampqVR/8IABAxAcHIyHDx9K7qSWn5+PxMRE9O7dG3w+v84P/hg6dCgiIyMlT5OprJ9GX18fmzdvxmeffYZVq1Zhx44dsLe3h5mZGfLz8/Hw4UOkpaWhf//+lY5WrAkHBwfMnz8fS5YswdSpU+Hq6gpbW1tJzVJTUxPLly+Hubl5nfdVW1OnTkVkZCQOHDiA6OhoODg44M2bN4iNjUW/fv0QHx9fq89n2LBh2LlzJ+7evQt/f3+4u7ujrKwM0dHRsLCwQO/evevl6nrChAlISUnBnj17JDeUaNGiBdhsNtLS0nD//n3w+XycPn0aZmZmdd5fRXl5efjxxx+xePFidOjQAba2thAIBHjw4AGePXsGXV1dye1dqyIeWObl5VXtVK/BgwcjOjoaR44ckdyKVBVqc/4CgC+//BLffPMNli9fjjNnzsDKygpJSUnIzMzEuHHjpJJ4TYiTs7h2LG8Ak7GxMdq2bYvk5GSp14iZmJjg448/xr59+zB+/Hh07twZJiYmuHv3LlJSUiTvt754e3tDR0cH586dw5gxY2BnZwc2mw0/P786zwSpiTFjxiA2NhZnzpzBwIED0aVLF+jr6yMuLg7p6emwsrLCypUrpV7TqVMnzJw5E+vWrcOECRPg4eEBCwsL3LlzB2lpaRg5cqTc6VdOTk5YunQpfvjhB8ycORN2dnb44IMPoK+vj6ysLNy/fx+5ublYuHAhXFxcavV+apSsHz9+XOm6ig8C+O677+Du7o69e/fizp07KC4uhrm5OUaOHImpU6fKDA766aefEBUVhTt37uDatWvg8/mwsLBAv379MG7cOLi5uUnKTpw4ETY2NoiPj0diYiLy8vJgamoKNzc3DB8+vNLpS5XR0tLCb7/9hjFjxuDw4cOIiYnBtWvXUFZWBiMjI3h4eKBXr14YMmSIZPQjUD6Mf+fOnVi5ciXCw8Nx5coVNG/eHLNnz8bEiROrnfqjCPE81IKCAmhpaUldzb+rffv2OH78OPbu3YuLFy8iMTERZWVlMDMzg62tLT7++OMqX19T48aNQ/v27REcHIz4+HjcuXMHRkZGGDhwIKZOnarwY98aipubGw4cOIDVq1fj7t27uHz5Muzs7PDtt99i/Pjxtf58DA0NcejQIaxevRrXr19HWFgYzM3NERQUhM8//xz/+9//6u09/PTTT/D398f+/fsRHx+P5ORk6OrqwtzcHAMGDECvXr3ktrTUVYsWLTB//nzcuHEDycnJePjwIdhsNqysrPDJJ59g/Pjx1Q7wE4lEkiZj8XzcqgQEBGDx4sWSOdc8Hq9e3ktt1PT8BZS/Ry6Xi23btuH+/ft4+vQpPD09sWHDhjrdac7BwQEGBgbIz8+Hnp5epfcncHd3lyRreQ/v+Omnn9C2bVscOHAAt27dgpaWFlxcXLBs2TLw+fx6Tdbm5ubYtGkT1q9fj/v370tGqFtZWSk1WbPZbKxevRo9evTAoUOHcPPmTfD5fNjY2GDSpEmYMmWK3PtsfP7552jdujV27NiBxMREaGpqwsXFBStXrkRycnKlc6UHDx4MJycn7Nq1C1FRUbh+/TrYbDbMzc3h5uYGPz+/OlWWWExdhq0SQgghpME16aduEUIIIY0BJWtCCCFEzVGyJoQQQtQcJWtCCCFEzVGyJoQQQtQcJWtCCCFEzanFgzwaq+zsQohEtZv5Zmqqjzdvav9kpvcNHQ9pdDyk0fGQ1liPB5vNgrGxXvUFiQxK1nUgEjG1Ttbi15P/0PGQRsdDGh0PaXQ8mhZqBieEEELUHCVrQgghRM1RsiaEEELUHCVrQgghRM1RsiaEEELUHCVrQgghRM1Rslay7PxSfLPhKtLfFKo6FEIIIY0EzbNWsje5JXiTV4q0rEK0MNFRdThNmkBQhsLCPJSWFkMkEqo6HCkZGWyIRCJVh6E26HhIU5fjwWZzoKWlAz29ZuByNVQdznuNkrWSsVjl/2cYuqGBKgkEZXj79jV0dQ1gYmIFDocDlvjDUQNcLhsCgepPxuqCjoc0dTgeDMNAKBSipKQQb9++homJJSXsBkTN4EomTgiUq1WrsDAPuroG0Nc3BJfLVatETUhjwGKxwOVyoa9vCF1dAxQW5qk6pPcaJWslE+cEEWVrlSotLYa2Nt2jmJD6oK2th9LSYlWH8V6jZK1kbHHNmu7rq1IikRAcDkfVYRDyXuBwOGo37uN9Q8layf6rWas2DgJq+iakntBvqeE1qQFmK1aswJ07d/Ds2TPk5ORAV1cXtra2GDRoEEaMGAFdXd0Gj+G/PmvK1oQQQhTTpGrWu3fvBp/PR/fu3fHJJ5+gX79+KCkpwdKlSzFixAgUFjb83Of/RoM3+K4IIYS8J5pUzfrmzZvQ0tKSWf7tt98iNEY14cYAACAASURBVDQUBw4cwKRJkxo0BnFjEQ0wI03R559Pxa1bcYiMjK3TdoKCBgEADh06UR9h1VlcXCxmz56GiROnYPLkz1QdDnkPNalkLS9RA0BAQABCQ0Px/PnzBo+BmsGJqnl7e9So/MGDx2FtbdNA0RBCFNGkknVlwsLCAAD29vYNvi9qBieqNnHiFJllBw/uR0FBAYYPHwV9fX2pdfr6BvW27x9/XIzS0pI6b2fNmo31EA0hjUeTTNabNm1CWVkZcnNzERcXh7t378LLywtBQUENvm821ayJislrpv3nn5MoKCjAiBGjGrQWbWVlVS/bsbVtXi/bIaSxaJLJevPmzSgqKpL8e+jQoViwYAE0NTUbfN80dYs0Jt7eHnBxccPChUuwZcsG3LgRhezsbPz66wr06OGLuLhYnDv3D27fTkBGRgYYRoTmze3Qp08/jBw5WmYuu7w+69OnT+B//1uE77//GRYWlggO3oLk5AfgcLjw8OiMWbPmwNZW+gJCXp/19u2bsWPHVqxduwnZ2dnYu3cXnj17Cl1dHXTr1gMzZ36JZs2aybzHkydDcfDgfqSkvICBgQG8vX0wbdosTJw4WmYftfH8+TPs2LEVcXGxyMvLhYmJKT780AsTJ06BubmFVNk3b7Kwb99uREVFIjMzA1wuFyYmpnB0dMLEiVMkF1IMw+DUqeM4fvwoUlNTUFRUBCMjY7Rq9QEGDRqCnj171ylmon6aZLKOj48HwzDIzMzEtWvXsHLlSgQFBWH79u2wsVG8VmFqql99oXcI2eUD8BmGgbl5/TUvvg+UeTwyMtjgctV7MoQq4uNwZI9Lfn4epk2bCD09ffj5+YNhRDA2NgSXy8a+fbvw4sULODo6oXt3HxQVFeHmzRhs2LAGDx7cw5Ily6W2JR6zUXEfbHb5sqioCERGXkG3bt0RGBiE+/fvISzsIh4/TsbevX/LvZiWt53Q0MOIjIxAjx4+8PDwxM2bMTh16jjS0l5i48ZtUq8PDt6KLVs2wsjICAMHDoGOjjYiIyMwZ84MCIUCcDhchT4HDoctiaFi+Tt3bmP27BkoLS2Bj09PNG/eAg8fPsDx40dx9WoEtmwJlrQSlJQUY8aMT5Ga+hKdO3+I7t19wDAM0tNf4cqVy/D374MWLcrLrl+/FiEhO2FjY4tevfyhp6ePrKws3L9/F+Hhl+Dv36famOsbm82mc1oDapLJGig/aVhYWGDIkCFo1aoVRowYgV9//RUbNmxQeBtv3hRAVMMqcnZu+S35yi8W8mv02veZubmBUo+HSCRS+YMQqqKqBzUIhbLH5fHjRwgI6I/58xeAy/3vlCEQiDB37jxYW9tI3RSDYRgsXboYp0+fwPDho+Do6CS1TvxaMfFvKCIiHH/8sQGuru6SdQsX/oALF84iPPwyevb0l4lX3nauX4/Ctm270br1B/+WEeCLL6YjPj4OCQmJ6NjREQCQkvIC27dvgampKbZv3wszMzMAwJQpMzFnzkxkZmbCyspaoc9BKBRJYhCXF4lEWLx4AYqKCrFs2Sp4e/tIyv/11x78+ecfWLZsCVavXv9v3DeQmvoSI0aMwuzZX0ltn8/nQyAok2w7NPQozM0tsHv3AWhra0uVzcnJUcl3RyQSVfsbZrNZtarkkCacrCtydnZGs2bNEB0d3eD7EvdZUzO4+rp6+xUiE1+pNAYWq+pBiN5O1ujWyVopsWhoaODzz7+UStRiNja2MstYLBY++mgkTp8+gZiYG1LJuiq9ewdIJWoAGDw4EBcunMX9+/fkJmt5hg8fJUnUAMDlctG//yAkJMQjKemuJFlfuHAWQqEQQUGjJIlaXH7q1BmYPn2yQvurzO3biXjx4jnc3DykEjUABAV9jMOHDyIm5gYyMl7DwsJSsk5LS/vdTUFTU1OmZYHL5YLNlq31GxkZ1Sluop4oWQMoLCxEQUGB3P6s+kZTt0hjY2VlDWNjE7nrioqK8Pff+3DlymWkpKSguLhIan1mZobC+7G3d5BZJk5i+fmKP9HJ3r59Fdv5r+aXnPwQAODk5CxTvkMHxzrfOz45OQkAZC5AgPJE6+zsglevUpGc/BAWFpZwdXWDubkF9uzZiQcPktC1azd06uSMdu14MrEEBPTDwYN/YezY4fDz84eLixscHZ1kRvKT90eTSdZPnz6Fubm5zJe5rKwMv/76K0QiEXx8fCp5df2hAWbqr1sn5dVaK6MOzysWMzU1k7tcIBBg9uxpSEq6h7ZteejTpy8MDY3A4XBQUFCAgwf3o6ysTOH9yEs04iQlbmZWhJ5eVdv572ET4jsWGhsbyy1vaFi3GmpBQcG/25d/oSM+roWF5eX09PSxefMObN++GVevXkF09DUAgJGRMYKCRmLcuImS9/Hll1/B2toGp06dwJ49O7Fnz05wOBx4eXXH7NlzaV78e6jJJOvw8HCsXr0a7u7uaN68OQwNDZGZmYmoqCi8fv0arVu3xjfffNPgcVDNmrwvIiLCkJR0DwMGDMb8+Quk1t25k4iDB/erKDLF6OmVPyI1OzsbdnatpNYJhULk5ubIjNauCfHFR3b2W7nr3759828c/11cWFhYYv78BWAYBk+fPkFcXAyOHDmIbds2gWEYyRx5DoeDESNGY8SI0cjOfovExFu4cOEcLl++gOfPn2L37gNyuy1I46Xew2HrkZeXF4YNG4asrCycPXsW27dvx8WLF2FjY4NvvvkGR48ehbm5eYPHIbkpClWtSSP38uVLAED37r4y6xIS4pUcTc21a8cDACQmJsisu3fvjlQtvHbbL2+Oj4+/KbNOKBQiMfGWVBwVsVgsfPBBGwQFfSwZgHblymW5+zE2NoGPjx9++WUZ3N098eLFczx9+qROsRP102QuvXg8Hn7++WdVh0EDzMh7w9q6vKsgISEe3t49JMsfPUpGSMhOFUWluF69+mDnzm04dGg/+vcfKGmWFggE2Lq17ndI69TJCS1a2CEuLhZXr0agW7fuknWHD/+N1NSX8PDoLOlPf/LkMYyMjGBiYiq1HXHNXDzqm8/n4+7dJHTsKD1wTyAQIC8vV6oseX80mWStLv673Shla9K4eXl1h42NLfbvD8GTJ4/Rpk1bvHqVhqtXr6Bbtx64fPmCqkOskp1dS4wfPwk7dmzFJ5+Mgp9fb2hrayMqKhJaWtowMzMHi1X7xkc2m40ffliIOXNm4vvvv0aPHuXzrJOTH+D69SgYG5vg66/nS8rHxt7A+vVr4OhYnuSNjU2QkfEakZHhYLPZGDVqPACgtLQUn302Cc2bt4C9fXtYWlqDz+cjNvYGnj17Ch+fnmjRwq7Ox4eoF0rWSsYC1azJ+0FXVxdr127Cxo1rER8fh1u3bqJFi5aYPXsuunTxUvtkDZTfetXCwhIHD+7HiRPHYGDQDN7ePTBt2ix89NFA2NrKDj6rCUdHJ2zZsgs7d25DXFwsIiLCYGxsgkGDhmLChE9hafnf7Vc7d+6K9PRXSEi4hcjIcBQVFcHExBSdO3fFyJFj4OjYCQCgo6ODmTNnIzY2FrdvJyIiIhy6unqwtW2Ob775HgMGDK5TzEQ9sRiq4tVabW6KUlwqwMzVVzBpUEd4d7Ss/gVNhLJvipKe/hxWVi2Vtr+aUqfR4OpA2ccjJeUFRo0ahl69/LFo0VKl7VdR6vj9UOQ3RTdFqb0mM8BMXdCDPAhRH9nZ2RAIBFLLSkpKsHbtKgBAjx49VREWITKoGVzJaJ41Ierj0qVz2L07GO7unWFuboHs7LeIjY1GRsZreHp2oQdiELVByVrJaIAZIeqjY8dO6NChk+SJWCwWC82b2yEwMAgjR46ReztPQlSBkrWSsSRTtyhZE6Jq7dt3wNKlK1UdBiHVostGJZM8nIhyNSGEEAVRslYyFt0UhRBCSA1RslYyScWamsEJIYQoiJK1krFY5bdFoT5rQgghiqJkrQIsFguUqwkhhCiKkrUKsFjUDE4IIURxlKxVgMVi1fg2pYQQQpouStYqwGaBmsEJIYQojJK1CrBYLBpgRgghRGGUrFWARTVr8p579SoN3t4eWLJkodTyJUsWwtvbA69epSm8LW9vD0yfPqWeI5QWFDQIQUGDGnQfNVGb40Teb5SsVaB8NDhla6IaCxf+AG9vD5w6dbzKcgzDYOTIofD29sCLF8+VFF3DoORHGjtK1irAZtE8a6I6gwcHAgBOngytslx8/E2kpr6Ei4sb7Ozq59nfn332OfbuPQRzc4t62V59WbNmI9as2ajqMAipFD3IQwVonjVRJVdXdzRv3gK3byfgxYtnsLNrJbfciRPHAAADBw6pt32bmZnBzMys3rZXX2xtm6s6BEKqRMlaBVhUsyYqxGKxMGDAEGze/CdOngzFjBlfyJTJz89HePhl6OsboGfPXigrK0No6GFERV3Fs2dPkJ39Frq6uujQwRFjx06As7OrQvtesmQh/vnnJA4ePA5raxvJcpFIhL//3ofQ0CN4/TodxsYm6NOnHyZOlN9XnZR0H//8cwK3bsXh9evX4PNLYW1tAx8fP4wfPwna2tqSst7eHpK/hw8fLPnbxcUNf/65BQAk/dWHDp2Q2k9paQn279+DCxfOIi0tFZqamrC374DRo8ehS5euUmVPnz6B//1vEb7//mdYWFgiOHgLkpMfgMPhwsOjM2bNmgNLSyuFjlNVzp07g6NH/8ajR48gEgnRsmUrDBgwGIGBw2Ue6RkffxP794cgOfkhcnKyYWDQDNbWNujatRsmTPhUUu7Nmyzs27cbUVGRyMzMAJfLhYmJKRwdnTBx4hSpz4qoBiVrFaCaNVG1/v0HYtu2jThz5jSmTp0JLlf6VHDu3Gnw+aUYOHAwtLS08eZNFtau/R2dOjmjSxcvGBoaIiPjNSIiwhEdfR1LlvwGb+8etY5n5cqlOH78KCwsLDFkyEcAgDNnTuHJk0dyy584cRSRkeFwdnaDp+eHEAjKcOfObezeHYz4+JtYt26z5D1NnDgFERHhePToIYYPHwV9fX0AqDYBlZWVYc6cz5GYeAtt2rRDUNDHyM/Px+XL5/HVV7MwZ863+OijETKvu3r1Cq5ejUDXrt4YMuQjJCXdQ1jYRTx6lIzdu/+CpqZmrY/Thg1rsG9fCExNzdC//0BwuRqIiAjD6tW/ISHhFhYvXiope+3aVXz77ZfQ09OHt3cPmJtbIC8vF8+ePcWxY4clybqkpATTp09GWloqPD27oFu3HmAYBq9fv8KVK5fh5+dPyVoNULJWAW1hCQ0wIyplamoGL6/uiIgIQ1RUJHr08JVaL+7PHjRoKADAwKAZDh8+KdPXnJmZgU8/HY/16/+odbK+dSsOx48fhZ1dS2zdugt6euXJdNKkqZgy5RO5rxk3biLmzv0OHA5Havn27ZuxY8dWXL58Af7+fQEAkyd/hvT0V3j06CFGjBilcOLZvz8EiYm34Ovrh8WLl0lqrWPHfoLJk8dh3brf4eXlLbO9yMgr+OOPDXB1dZcsW7jwB1y4cBYREWHo1auPYgfmHYmJt7BvXwiaN2+B7dt3Q0/PAAAwdep0fP75Z7h06Ty8vX3Qp0/5+z558hgYhsG6dZvRrh1Pals5OTmSv2Njo5GWlooRI0Zh9uyvpMrx+XwIBGW1ipfUL0rWSsZPT8fEO3twv8WnANqoOhwiR17UVeRGXlFpDNXNGDD07oFmXt3qtI/Bg4ciIiIMJ0+GSiXrpKT7SE5+CHt7B7RrZw8A0NTUlDsozNzcAj179sKhQweQnp4OK6uaN/P+889JAMCECZ9KEjUAGBgY4JNPJslM/wIAKytrudsaPvxj7NixFTExNyTJurZOnToONpuNGTO+kGpetrVtjqCgkdi5cxvOnfsHn3wyWep1vXsHSCVqoHxQ34ULZ3H//r1aJ+vTp8ub6D/5ZDIMDQ0hEIgAAFpa2pg27XN88cV0nD59XJKsxbS0tGS2ZWRkJLNMS0tbZpmmpmadWgJI/aFkrWTCoiKwAGgUFag6FNLEdeniBQsLS9y4EYWsrEyYmZkDkK1Viz1+/Aj79u1CQsItvHmThbIy6RpXVlZGrZJ1cvJDAICTk2y/97tJT0wgEODo0UO4cOEsnj9/isLCQqmLm6yszBrHUVFRUSFSU1/C2toWNja2Muvd3Dywc+c2JCc/kFlnb+8gs8zCwhIAkJ+fV+uYHj4s35erq4fMOmdnV3A4HKl4+vTph/Dwy5g6dQJ69fKHm5sHOnVylsQi5urqBnNzC+zZsxMPHiSha9du6NTJGe3a8WRaLojqULJWMhb33y+/kJqW1FUzr251rrXWFZfLltScGgqbzcaAAYOxY8dWnD59AuPHT0JpaQkuXDgDHR0d+PsHSMreuZOI2bOng2FE8PTsAh+fntDR0QWLxUJ8/E3cuhUHPp9fqzgKC8svXE1MTGTWGRvLLgOAn376DhER4Wje3A4+Pn4wMTGV9FHv2LG11rGIFRRUHhNQ3o1QsVxF4j7xisRJTySq/Wda1XHicrkwNDRCbu5/zds+Pn5YseIP/PXXHpw6dRyhoUcAAA4OHTF9+iy4uZUnfT09fWzevAPbt2/G1atXEB19DQBgZGSMoKCRGDduIiVtNUDJWslY/55QWHX40RJSXwYMGIxdu7bj1KnjGDduIi5fvoiCggL07z9Iqkl69+5g8PmlWL9+q8zI7xUrluDWrbhaxyDez9u3b2Vq5tnZb2XKJyXdQ0REODp3/hC//bZGKpFkZWVhx46ttY5FTJxw376V3T9QPnq6YjllqHicdHWl+8kFAgFyc3Nk4vHy8oaXlzeKi4tx794dREVF4OjRw/j66y+wc+c+yfx5CwtLzJ+/AAzD4OnTJ4iLi8GRIwexbdsmMAxT6ah8ojx0UxQlY3H+vT4SClQbCCEo7/v18OiC1NSXiI+/WWkT+MuXKWjWzFAmUYtEIty+nVCnGMSDnxIT42XWxcfflFn28mUKgPJE9G6NLyFBdhsAJH3OitZsdXX1YGvbHBkZ6UhLS600rrZteTLrGgqPZ//vvmNl1iUm3oJQKETbtvZyX6ujowN3d0/MmjUX48dPBJ9fiuvXo2TKsVgsfPBBGwQFfYzVq9cDAK5cuVyP74LUFiVrJWNp/FuzFgpVHAkh5QYNKr/pydatG3HrVhxatfoAnTo5S5WxsrJBfn6ezFSq3buD8fTpkzrtv2/fAQCAXbu2S5p6gfK53rt2BcuUt7Iqr1XeuiWdmNPTX2HTpj/l7qNZM0MAwOvX6QrH1a/fQAiFQmza9KdUkk9LS8WhQwfA5XIRENBf4e3VVb9+5XPBd+0KRl7ef33fpaWl2Ly5PLEOGPDf/c1v3YqDQCBbKXj79g0ASOaiP3nyWLKsInGrRsU560R1qBlcyf6rWVOyJuqhe3dfGBubSGrIAwcOlikzfPhIREdfw/Tpk+Hn5w9dXV3cvp2IR4+S4eXVHVFREbXev6urOwYNCsSJE0cxbtxI+Pr6gWGAsLCL4PHs8fLlC6nyDg4d4OjohLCwi5g+fTI6dXJCVlYWoqIi4On5IV69kq0Ju7l5YP/+EKxY8T/4+vpBW1sbVlbWkgsFeUaPHo/r16Nw6dJ5pKQ8h6fnhygoyMelSxdQUJCPL774Wqnzj52dXTBy5GgcOLAPo0cHwde3FzgcLiIjw5Ga+hK+vn5SI+D/+GMlsrIy0KmTM6ysbKChwcWDB0m4eTMG1ta28PPzBwDExt7A+vVr4OjohBYt7GBsbIKMjNeIjAwHm83GqFHjlfYeSeUoWSuZpM+amsGJmuByuejXbwD27QuBhoYG+vYdKFOma1dvLF26Ert3B+PChbPQ0NCEo6MTNm8OxpUrYXVK1gDwzTfzYWfXEqGhR3D06CGYmJgiIKA/Jk2aCj8/L6myHA4Hy5b9ji1b1uP69Sg8eHAfVlbWGD16PEaPHo/Lly/Iib8bZsyYjePHj+Gvv/ZAIBDAxcWtymStqamJNWs2YN++EFy4cBaHDv0FDQ0NtG/fAR9/PAZdu3rX6T3XxqxZc9GunT2OHTuEU6eOQygUwc6uJb744msMGzYcLBZLUnbcuAm4cuUyHjxIQmxsNFgsNiwtLfHJJ5Olbg7TuXNXpKe/QkLCLURGhqOoqAgmJqbo3LkrRo4cA0fHTkp/n0QWi6G7c9TamzcFEIlqdvhEpaV4NPMzPHHsib5fyr/hQ1Nkbm6AzMx8pe0vPf05rKzq5+EUDUEZo8EbEzoe0tTxeCjym2KzWTA1Vd6gvPcJ9VkrGevfATEsEdWsCSGEKIaStbKJkzU1gxNCCFEQJWslY7FYELLYYAnVqwmLEEKI+qJkrQIiNoeawQkhhCiMkrUKiFgcuoMZIYQQhVGyVgERmw2WiOZZE0IIUQwlaxUQsTh0BzNCCCEKo2StAuV91pSsVY1uMUBI/aDfUsOjZK0CIhY1g6sam82BkFo3CKkXQqEQbDY9RrMhUbJWAYbNAZuStUppaemgpKRQ1WEQ8l4oKSmElpaOqsN4r1GyVgEaYKZ6enrNUFSUj4KCXAgEAmrGI6SGGIaBQCBAQUEuioryoafXTNUhvdfoQR4qIKKatcpxuRowMbFEYWEe3r5Nh0jNPg82m63ws5ebAjoe0tTleLDZHGhp6cDExBJcroaqw3mvUbJWARGbAw7dblTluFwNGBqaqjoMuZT9YBN1R8dDGh2PpoeawVWAYVHNmhBCiOIoWasAw+aAzai+CYsQQkjjQMlaBWieNSGEkJqgZK0CDJtNzeCEEEIURslaBWieNSGEkJpoMqPBs7OzceHCBYSFheHhw4d4/fo1NDQ0wOPxMGzYMHz00Udgs5Vz7SLicKnPmhBCiMKaTLI+c+YMFi5cCHNzc3Tp0gU2NjbIysrC+fPn8eOPPyIiIgJr1qwBi8Vq8FgYNgccUfmNOJSxP0IIIY1bk0nWrVq1wsaNG+Hr6ytVg547dy6GDx+Os2fP4ty5cwgICGjwWAqamUFDWIaSJ4+h06Ztg++PEEJI49Zk+qy7du0KPz8/maZuc3NzfPzxxwCA6OhopcSSYdMeZWwu8q5FKWV/hBBCGrcmk6yrwuWWNzBwOMp5agyjoYm3euYofZmilP0RQghp3Jp8shYIBAgNDQUAdO/eXSn7ZLFYyNUxAj/9lVL2RwghpHFrMn3WlVm1ahUePnyIHj161DhZm5rq12qfOtoayNU2gijjPoy0AI1mBrXazvvG3JyOQ0V0PKTR8ZBGx6NpadLJevfu3QgODkbr1q2xfPnyGr/+zZsCiEQ1f7RiaakAfC0jAMCru8nQaduuxtt439CDCaTR8ZBGx0NaYz0ebDar1pWcpq7JNoPv2bMHS5YsQZs2bRASEgITExOl7ZvFAgo09QAAguxspe2XEEJI49Qka9Y7d+7E0qVLwePxsHPnTpiaKvcxiSwWUMouf/arqLRUqfsmhBDS+DS5ZL1lyxasWrUKDg4OCA4OVmqNWozFYqGMVX7oRXxK1oQQQqrWpJL1+vXrsXbtWnTs2BHBwcEwMjJSSRxsFiTJmqGaNSGEkGo0mWR99OhRrF27FhwOBx4eHggJCZEp4+DggN69ezd4LOU1aw7AYlEzOCGEkGo16mSdnZ0NY2Njhcq+fPkSACAUCrFr1y65ZQIDA5WWrBmwwNLUpGRNCCGkWo0iWf/999/Iy8vDp59+CgB48OABpkyZgszMTDg4OGDz5s0wNzevchuzZs3CrFmzlBFutVgsQMQwYGtpUTM4IYSQajWKqVshISHQ1taW/HvZsmVo1qwZvv/+exQUFGDt2rUqjK7m2GCB+TdZU82aEEJIdRpFzTotLQ0ffPABACA/Px8xMTFYv349fHx8YGRkhN9//13FEdZMec0aYGlqQVRaoupwCCGEqLlGUbMWiUSS5z7fvHkTANC5c2cAgLW1Nd68eaOy2GqDxfq3Zq2tDaaUr+pwCCGEqLlGkaxbtWqF8PBwAMCpU6fg6uoKHR0dAEBGRgYMDQ1VGV6NaWqwUcoXlg8wo3nWhBBCqtEokvWkSZOwa9cudOnSBSdPnsS4ceMk665fvw57e3sVRldzFkY6EIoYlLG5EJVQMzghhJCqNYo+60GDBsHa2hqJiYno1KkTPD09JevMzMzQq1cvFUZXc5YmugCAUnChTTVrQggh1WgUyRoAPDw84OHhIbN89uzZKoimbiyNy5vwi0RsaNJocEIIIdVoFM3gcXFxuHz5suTf2dnZmDt3LgYNGoTly5dDKBSqMLqaa6anCR0tDgqFbIhogBkhhJBqNIpkvWrVKty9e1fy7xUrViA8PBytWrXC/v37sWnTJhVGV3MsFgvWZvrIE7DA8EvBiESqDokQQogaaxTJ+vHjx3B0dAQAlJWV4ezZs5g/fz7WrVuHL7/8EidPnlRxhDVna66PLIEGwDAQ5uepOhxCCCFqrFEk66KiIujr6wMAEhMTUVxcjJ49ewIAOnbsiFevXqkyvFqxMdNDeln5M60FOTkqjoYQQog6axTJ2tLSEklJSQCAK1euoF27djA1NQUA5ObmSt2KtLGwMddDPqd8oBkla0IIIVVpFKPBBwwYgN9//x3R0dEIDw+XeiDHvXv30KpVK9UFV0s2Zvoo4P6brHMpWRNCCKlco0jWs2bNgpaWFhISEjBlyhRMmDBBsi4pKQl9+/ZVXXC1ZGKojUKODhiwIKSaNSGEkCo0imTN4XAwffp0ues2bNig5Gjqh7GBFkQsNoTautQMTgghpEqNIlmLPXz4EDExMcjJyYGRkRE6d+6Mdu3aqTqsWtHgcqCrxQVfW4+awQkhhFSpUSRrgUCAefPm4dSpU2AYRrKcxWJh4MCBWLZsGTgcjgojrB1DfU0Uc7UhLChQdSiEEELUWKNI1n/++SfOnDmD2bNnY/DgwTA3N0dmZiaOHz+O9evXo0WLFo3ytqPNdDVRxNaCsIDmWRNCCKlco0jWx48fx7Rp06T6rW1tbTF9+nQIhUIcJec9hgAAIABJREFUOXKkUSZrQ31N5EMTwvx8VYdCCCFEjTWKedYZGRlwc3OTu87NzQ0ZGRlKjqh+NNPVRB6jAVFRERiBQNXhEEIIUVONIllbWFggLi5O7rq4uDhYWFgoOaL6YWqojVxoAgCEhYUqjoYQQoi6ahTN4IMGDcKmTZvAYrGk+qxPnz6NTZs2YcqUKaoOsVbsLPSRwC6/+5qwIB9cQ0MVR0QIIUQdNYpkPWvWLLx8+RLr1q3Dn3/+KVnOMAwGDhyImTNnqjC62mthaYBijhYAoOxNFjQsLMDW0FRxVIQQQtRNo0jWXC4Xq1atwrRp0xATE4Pc3FwYGhrC09MTmZmZCAwMxIkTJ1QdZo3p62hAo5kBkAakrf0Dei6usP38C1WHRQghRM00imQt1q5dO5mboDx58gSPHj1SUUR1p21ri6xUS5jlv0bhrXhVh0MIIUQNNYoBZu8zC1N97G/RD3qubmDr6Kg6HEIIIWqIkrWKWRjropAvAqtFK4iKiyEqKVZ1SIQQQtQMJWsVszQur00XaugBAATZ2aoMhxBCiBpS2z7rlJQUhcplZWU1cCQNy9JEFwCQzdKBEYCy7GxoWtuoNihCCCFqRW2Ttb+/P1gsVrXlGIZRqJy6MjPUBosFvBZqwQiAIPutqkMihBCiZtQ2WS9dulTVISgFl8OGmaE2XpWwYA9AmEf3CSeEECJNbZN1YGCgqkNQGktjXbzK44PF5UJYSI/LJIQQIo0GmKkBC2MdZOSUgKNvAGEB1awJIYRIo2StBqxN9VBcKgCjowthAdWsCSGESKNkrQbaNS9/gEcxV5uSNSGEEBmUrNVAcwt96GpxkSfiQkTJmhBCyDsoWasBNouFD2yaIVvIpZo1IYQQGZSs1YS1qR7elHEgLCwAIxKpOhxCCCFqhJK1mrA21UUBSwtgGAjz81Cc/FDVIRFCCFETlKzVhLWpLrI1DAAA6du2IGX5/1D8WPrRn6/3hiDvxnVVhEcIIUSFKFmrCRszPWRoGQMAiu7fK///vbtSZXIvX0T61k1Kj40QQohqUbJWEwa6mjCwNAefqyVZlh8TDUYoBADqxyaEkCaMkrUaad/SGKlaZgAA3Q4dwU9LxdN53yB13R8QlZaqODpCCCGqQslajbi0NUOoRTdkj/sStnO+BkffAILstyhMuIXHs6arOjxCCCEqQslajXRsbQJjCxNcfikCi8UC18RE1SERQghRA5Ss1QiLxYK9nTFSMwvAMAwla0IIIQAoWasdaxNdFJYIkF9UBra2ttwyjECg5KgIIYSoEiVrNWNtpgsAePWmEGDklxEVF9dpH4LcHOTfjK3TNgghhCgPJWs1Y22iBwB4+iofGuZmcssIi4rqtI+Xv6/Eq41/QsTn12k7hBBClKPJJOvQ0FD8/PPPGDFiBJydnWFvb49169apOiwZJs20wGtuiNCrT6Hh1w+mQ4fJlBEVV56sRXw+8m5cA8NUUi0HwE9LBQAwlKwJIaRRaDLJes2aNfjrr7/w7NkzWFhYqDqcSrFYLIz256GUL0T842wY+fWSKSMsLAQAFCTcQtHDB1Lrsg4fRPrWzShOul/5Tv5N5KKysvoLnBBCSINpMsn6119/xcWLFxEdHY3p09V7znILC31Ymugi9kEG2Dq6MuuFublghEKkrfsDL1cslVonyH5bXkaBR21SzZoQQhoHrqoDUBYvLy9Vh6AwFosF5zamuBSXCoFQBLNhQdC0tgGnWTOkLP0Vpa/SkLP8f5LyjEAAFrf8oxT/nxFUX2umZE0IIY1Dk0nWjU37lsY4F5OCx6l5aN9/oGQ5W1cP2f+ckipbmvoS2i1blf+DwwGgWM1aVEbJmhBCGoMm0wze2PCaG4HFApJeZEst5zQrf4ymYQ8ftJj/I/D/9u47PKoqf/z4e/pkMpPeCSkEgkAAkRI6KogiuoCKYltXv7trW9vadVdx1bX/FsGyui5rB0WKDREbvQeUToB0kkz6ZHq9vz8muWRIKAKShJzX8/A8mXvPvffcwySfe8o9B3AVFQLBvuxAU5D2W63HvYaoWQuCIHQOomZ9CmJjjad0fHy86Zj7s1KjOFhhDUlX0PTaVlLuYGJzz6XCZIKKMuLjTay/81Z5wQ+1xxlynCRJWLbvIDKnH/lN2yIMaqKPk4cz6Xjl0dWI8gglyiOUKI+uRQTrU1BbayMQOPorUscSH2+iuvrYtd+eyRF8n1fKofIGtJpg87avsREAT1wKNTU2tOkZ1O/Yxc7/NydkZS57dW3I+Z0FBZT+8x9EjBwtb6uvbsB3nDycKSdSHl2JKI9QojxCddbyUCoVp1zJ6apEM3gH1r9HDD6/xE/bDsnbkv74Z/RZPVHHxAKgz+yB11yJZeWKkGP91saQz77aGgAa162Rt0ke8eqWIAhCZyCCdQd2Tno0OT1i+Hp9Mf5AAICI4SNJe/RvKBQKAKIuGI+hb79Wx3oqzSETozS/0tWSGGAmCILQOYhg3YEpFArGDkjB5vRyoMzSZhp1RASpf30QdVzo1KQBhx1vdTV+hwPzB+/hLj/U6tjmmrUkSXjrWgdzQRAEoWMQwbqD65cZg1ql5PM1hbi9/qOmixp3gfyzMiwMAFdRAZYVP2JZ+RONa1a3OkbyBPu465ctpfChv+Ktrm6VxltXe6q3IAhH5SopFnPUC8IJ6DIDzBYsWEBeXh4AxcXFAHz//fccOhSscQ4ePJjp06e3W/6OJkyn5qZLejN36R7eWLyT+64e2Ga66EsuJXLMOJz789FnZlL4+KNUvv3vkDRKg4FAi0VAmqcbtW7aCIC74hAecyW2n7dhGpZLwOmkfM4sut3zV8L7D/iN7lDoqvw2GyX/eBLTsFyS/9yxZxUUhPbWZYJ1Xl4eixcvDtm2d+9e9u7dK3/uiMEaYFT/ZKobnHyxtogGm5soo65VGoVCgcpoxDjoPADC+w/AtmVTSBp9eiaOPbvkz83vWTevj924dg2+hgZcBw9gWfkTuqaJVtyHykSwFk67gCu41KvjWPPYC4IAdKFg/fzzz/P888+3dzZO2qBe8Xyxtog9xfWM6Jd03PSxky9vFaxjLv/d4WCtUlH/7Td4q6vwVJQDYGta41qf1RNXwUHcTZOtBNxubNt/xrphPUl/uk0e3HYipECA+m+XEXXBBSj1YSd8nHD2C7hcAEj+o3fvCIIQJPqsO4nuiUaMYRo276k6ofS67t3p+ebbwQ8KBcm3/wVDdu/DCZr+QNq25rU6NqxnL/RZPeXPvtoaymfPwrppI96a1v3ax2L7eRs1Cz+leuGCVvu89fVUffyhXLMH8FRWYNu29ZjnLPvXyxT97dFW2yWfT66tHU3tV19g/uj9E8z9EfmtrZEH6kmShGPvnmMuRSocW8DZFKx9IlgLwvF0mZp1Z6dUKJgwOJUlawoprGgkMzni+MdotHR/+HE08XGoo6KPmi7lL/egTUyk+KknkHw+NPHxGPrlYNuyCevmTTSuWyunte/Yjjc+AUOfvlS+NxdVmIGE626Q99d++TmqiEjUEREYB52H1NQv7m9sbHXdqvf/h33HdsLPHQTJwwEo+efTBBwOer7xNkqtts38OnbtBII1fsnnQxUeDkDFO29h27KZXv/531Fr/7VLFgGQeP3vW+3z1tdT/PdHSfq/P2McdB71y5eh0GiJuuBCAAoffiBYruHhRF90MbVLFqGJjyf+muswnjuozetJkkTD98sJH3Au2sTENtN0Vf7mddn9vmMnFE6bgNuNQq1G0bSGgNB5iGDdiVw0tDvfbSll8eoC/nr1uSd0TFivXiGfu933ACqTiZJ/PAlA+ICBGPr1Q6nREtb7HBy7dqIyRRDetx/hffvhPHgQT1OftX3Hdqo//hAAhU6H1DRjWuzUaUh+PwqFktrPD48L6PWf/x2zqdPXNH+55PPis9vx1tXJA+DcxUWE9crGW1eL+f33CDgdKLVakm+7Uz6+/LXZOPbsIvEP/0fEqNHYtmwGwG+zojYdfpiRJImqD0Nr09ZNG/Hb7XIgBnAXFRJwuSh/fTY93/wP1Z/OB0ATF4fffnhhlIDdTsNPPwLgra6m/LVXyX7n3TbL31dXR/Un83CXlpB0y58AqF64AH1mD0znDUYKBAi4XKgMrZdCPds1t4KIZvAz58CdtxI+6Dy63Xl3e2dF+JVEsO5EwnRqLh2ezoIVB8kvbSC7e9SvPkd4vxwAYiZfDkolcVOmyfuixl+EY9dO9BkZ8rboiy6mYcWPJN/+F2oWLcCxezd+mzWkpmzdsJ6aRZ/JgblZ4cP3YzinD9B2U2fzq2N+i4XtDz2Gs6xM3uc8cICwXtnUL1+GY+d2eXvN4oXyz8397+Z3/4smNlbeXnDf3aQ/+TTu0hIc+fuwbc0j4LCHXLvi7TcBiBx3Pu6y0uCo5BGHl1Gt+ezTkLQBZ2jzur+x7ffej+TM3weAbdvW4Oh7SZJXTdM99yKNa1djWbmSHq/MwrLiR1SmCExDh53QuTsqT2Ul5W/MIfX+h1BHRh41XXMzOGewK8FTWYHf4QBJIqxFV09XIDVNrGQ/TjeT0DGJYN3JXHheKt9uLuWTH/fz2I2DUSlPbthB3LQrW20zDhjYqoYYOXoMkaPHAJAw43oAfBYLZf/vJeKnX03NksVUNdW2j+Srq5Ob0P220HmM/U4n3trgRCyOfXtDAjWAY+9uoi++BFfBQbRJyaiiovBUVGBZ+VNIuox/vkjRYw/R8NMPIduLn/q7/LMyLIyEG37fqnYNwZqxdeN6AKzr16HQ6TD06UvDD9/Jadqs+R0RYCRJQqFQUPvFElSmCKIuuJDGdWupnPsfAAJOJ47du1BHH+6OcBUcxLJyJX5rI/v/fIu8vbMH67pvl+IpP0T1J/OIm34Nmui2u2ACzsOvETaX32/JVVREyTMz5c/Z77wbnBCouhptQsJveu2O4HjjOYSOTQww62R0WhXXju9FYYWVl+f9TKAdBjipIyPJeOoZwnMGkDDjuhM6xl1USMV//k3VvI+o//F7Sp9/FskdrFlZN6wPSatNSsa5by/FT/4NV0EB4f0H0P2Bh4mZfFmrIKlNSECXkdnmQLlmKXfeTdT5F2LKHd5qX/Vnn1D/7TL5syYunvABoe+yn8hSotYN66mc+w61Xyyh6qP38TU2UjX/YwDUMbEoDeFYN2+UR94DWFb81GoOdyBk0Jqz4GCwJtgGn7XxV7/25GtowNFU22913aaa14kKeL1UzfuIQ3Nm4alqMfCxKf/WTRvk7pY2j28RPAJOZ6uWmdOl+rNPqf/+O+q/W9ZqX+PaNRQ99hDO/fltHHliXMVF+Cwn1tLSngL2tr9HQucggnUnlNs3kd+NymBfaQNFFe278o6+jabEHv9vdptprRs30PDDd1R//CGeQ2Xoe2S1mS7u6hlIPp8c2JqDZ8tmS3V0DN3++iBwuCaqjo0l8vzDfdDx11yLoW8/wnoG++2Tbv4jqQ88TMpd92I8bzDQuklQEx+PccBAFOq2G50UajVhLUbV63v0AKDyv2+HLJJyaPa/CDjsxF05nbTHnyB8wAAcu3YF70mpRKHV4tyfj7JpcBxAfHPLRdM87n67ndJ/Pk3lO28Fg//HH1I59x2secG++fLZsyh7+QVqv/qC/XfeFjKqHoJB/8gAWPLc05S9+JwcmD2VFUiBAJa1q9n/51vkVd1OROPa1TT88B32X36m7usvcTdNMESLoN/Ww0gzuRm86VwH7rmz1Sx6Abeb6k/n42k4fjD022xUvjsXf4suC0mSqF+2lOr5H+HYtzckvRQIYN+5Azi8Jrzk81H7xZITroUG3G5Knp5J4WMPHfcthpZcRYX4bbbjJzyN/Ed0Bf0a1Qvmc/C+u8XbD+1ININ3UhOGdOfLdUVsP1hDj5Tjjwz/rSgUClLuvAu/3Y753bkAqEyh6+wacgaQeP2NBLxe6r5cgnXzJlCpSHvs7+T/8Q8AZN1+K+7wKDSJSaijooidMi04KOv//oxSF5wERtc9DV1aOu7SEtKfeEq+TvRFFyN5vRj69CUsqyfR4yfg2LeXqPMvJPqiiw/nVa2W+9CNA8+lbulX1Cz6LCSv+vQM1FHRZM15E8nl4uB9dwGQcP2NeMxm4q+6Gk9FhdzM3v3hx2n44XtUJhOV/31bPo+7qBDj4CHETJoMQFiPLKwb1mNZuRJdWjoBmw1vTTWRI0dT/923TffXHQDz++/C8CFUbd4GgH37LxT89fCAoMZ1a9A88RSuwgLg8Ah369YtWFatJHLMOHz1ddR98zUBp5PMf76AJi4eAF9tcPpYn8WC32Kh5JmZxF99LdULg330tm1biRp3/uFrbVyPLrU7um6prf7vPeXlKPV6FGoNjWtX07h2NT3ffPuoLQEtuQ8dCulqqFv6Nfj9OA/sRxUVieT2ULPoM1QmE/XLl1GmUWCadk3IORz79qJLS0fVNL1u/fJlNK5ZhS4lheiJlwTvs75eTu9vaAg53m+zyWMP3GVlSIEA1s0bg8Ha6ST+mmuPex+OPbsBkNzu4MDE19+Sv69HE3C7KX3+WYznDT7qzG2SJNG4bg3GAcGBpJbVK4m+5FIkny/4hkW8CXdpKTWLFpBww00hYzaOet0T+H9p5rM0IPkDaGJiAOTWJ3dxEfqMzBM+j3D6iGDdSRnDNPRKjWLjbjNTRmf+5v19x8zLoGAt1dC3HwG7HYVCQfeHH0cdE4O/0YIqMkr+pU+86Ra8dXXENg1sS7jh93irqki6ZGLI+ryxl09pdR2FUkna32cGR0+HhYVsj73sd/JnbXIK2uSU4+Y75tLLiBp/EQqNRu4zDs/pD4BSowGNRk6rz8wi6oLxwfN36wZA9KTJKFQqoicGHwhaBmtArr0D6DKCNXC/tZHkW2+n7OUXgtcbdB76rCxUEZHoUrujiorCsXMHhU01vqNpq3m5ftk3uEuKcTY1jSsNBggEcBUXycG6mevAfrlZuP7H7w6/d78tDxTgq6kh+qKLqfzPWwBkvfo6llUrsW7eSNrjT6BQqfDW1qCOi0ehVstjEhrXrMZ3xKIwVR9/QPy1N4R8R2u/CJ1NsLkGblm9MtjPf0QNrnFvPi0fAZ3791P20vMYhwwj5bY7kAIB+VUwyedD8vvxmM0hrR1HMn/wLq6Cg8Hzr12NJi4OdVRw0OaJNmvbm14jbOYqLgqdz6ANjn17kXw+bFvz8FkbQ95c8DU2UvrCPzENGUrd11/iyB1OwOnEvv0XwnplU71gPq6CAuI+ek9+YLT9vBXn/ny0CYlou3UjIndEyPWaW1z89sM164DbjUKrbfPvhru0lOKn/o4mMYnuDz2KosXvgW1rngjW7UQ1c+bMme2dic7K6fSc9EDW8HAdDsepLWCgUipYtb2CrG6RJEa3/6s/qjCDPPpXExuLymBAHR0dGljVaiLHjEUbHxzQo8/IJLxfzgmXh0KhCAbS00ShVqNQKJD8fpz780m48aaQP2ABpxNXwUHipl0p15gUCgUxl08h/IilSXXpGQRcTvx2O5LHQ9yV01EZjQCojEbqv/8O0+AhxFw8CW99He6SYhJ/fzO61O5oYuNQajREjR1H+LnnoVOBo6hYPnd4/wFoU1LwVlaiy8jE39CAOjoadWxcsHlTklq9y24cMhRPWSm61O44du2k6pN5chpb3ma51hlwOFBFRRE5cjTWLZux/7wt2ESv1+NsajquX7YUx55d+C0Wwnr2RGU0UvfVl2iTkoKBuzrYZ+08sB/fEYu/uAoLsaxagddslmtsts0b8TUEr680GoPjAlQqfG1MuqPP6onz4EEM/XLkh76qD9/DW2XGU36IqAsnUP7m69g2bQBAm9od18EDVL71Bq4D+0PO1f2xJ3Dm7yVgt+OtrIBAANPQYXjKD+G3NqJNTsGxexea+HjC+we7Q9zFxagiItoMbA3fLQ+ZKMhVUEDEqNFH7UYBaPjpB1zFxeD3o9Bq5RaC+mVLKZ/9LwJ2m9yHrjIY5Nq74ZxzaPgxOJDSWV6BsyzY7eDYuQNPeTnO/fnYtuZh6NMXdUwMzn17saxeRfmcWcHWm4AfT1NXRd3Sr5C8XsJ6ZaNQqYJdLs8/iyY2Fuf+fBx7dhOw26hfviwkD837mged/loKhQKDoe35E4RjU0iiE+Kk1dbaCAROrvji400hNcmT4fX5efw/G1GrlDz9x2EnPTK8Izgd5XGq2hqRLAUC+Boa5CBxIjzVVdi2bCH6kkkh5/OYzahjolFqtEiBAJLXe9Qm0/h4E+V7Cqh8dy7OvXtI+tOtmIbmBvuDlUoCbrf8EOSzNlJwX7CZPHzAQHyNjbiLCom/+lrqv1+OJi5OfoWsJUPOAPm1uKgLx2PKHUHpc8+EpFFFRpF40x8onz0LpV5PwOVCk5iIr7YWyecjavxFuEtLcObvI3baldS2eLXueNRxcXS7+z7M7/0Pd2kJkaPHyMGoWer9D6HvkUXxYw+h6Z5G6r33IwUCHLznTrTJKbgKDmLo2w/H7l2tL6BSkfynW3EdPCh3NWS/8y6eyoqQGfAynn0B29Yt1CxcgGlYLtZNG1GZIvBbG1GGhRFwOkm48Q+oIyKoWbiA7o88Lj+EFT7yILqMDML7D6Ru6Zd4zWbir7sBAhKSx402pZs8YY4UCFDx1hvY8rZg6JdDwO1u9TABwbcXJL+/1cDGmEsvo27pV/Ln2KlXUP/9cgJt9H2bRozEun7dcf8PlEYjmc88j3XzJqqOMbOf0mgkIneE3HWRNft1VIbwo6Y/6nmUCmJjjb/6OEE0g3dqGrWKqy/oyRtLdrLjYB3n9oo7/kHCUbVVc1Iolb8qUANo4xOImXRp6+0tZjBTKJUojtO3qYmLJ/nPt1P/7VKM5w1BoVRC0wNZy9YKVbgxuD0QQJuULDd3ahIS0CQkyM3iR+p2z32U/ONJ3KUl6Htkoc/sEQy+h8oI75dD3Tdfk3TzLYTnDCDt8SdQR0dj/uiDkEF5CrUadVN/aXi//gRcLtylpcRPvxpvbQ1+qw3jeYORPG48ZjMEAlS88xb+hgb0GZnoUroRPWEiPmsj4f0H0PDjD8FxCSXBVgVDn74AJF92KSUfzcNdXo5CEWzxiBw7LlgzbStQA1EXTsA0ZBj6rF7Uf/ctsVOvCJZXROi735qEBIznDqJm0WfyCnTNzfLN79dXffCunL5s1isotVpip0zDW1eLKXc4kaNGE5E7nIIH76Pm0/khg/1MucPRdU9D3yNLnn/f0Kcv+h5ZlL34nJwuctwFRI2fgFKvx1dbS8U7b+GrrSV8wECc+ftCAjVA9MRLsKz8iZZj+BNuvAn7ju0hb1godDoickdgWbUiJE/WjRsI2GxUf/YJziMeGvRZPVFHRspvWZgGDw1p/vaazagye7RZ7sJvQwTrTu7cXnFEhmtZvrmEgT1j27XvWjj91BERxE+fccw0iqZADcFBeAqdDtfBA2gSEoi5ZBL21FQix15A8ROPoVCribt6BprYuGCXQtPMafqMHiiUShKuvV4+b/TFk4LnBvRNf5jDemRh37YVfVZPwnr2JOrC8SjDDIT37YcuPT1kQp2QQWlhYagjg/3B3e4OPiQ0jyto+V55t3vvR9stFeee3fLYAICkiy+iZP6nFD/xmLxNn5mFyhSBc+8ewgedhy45BfuunbiLiwAw9D4HAE10ND1eeRVVRLBvWGUwkPTHP6NQqUGpQKFQoE1OIXLMWCyrVrYqX1VEBP7GRtTR0fitVnmBm7KXggsDaZq6dBRqNeEDz6VxzWqMg4cER4cHAlg3bsC6cUPIOcMHDESX0o20J56i/LVX8dXVETP5MjQxwQcfTUws4f0HYlnxIzGXXka9RiMH+mZKrRa/NbQ1qrlbqfDnbfI2hVJJ+LnnBoO1QkGvt+ci+bxok5LxVJTLa92n3H0vkstNxdtvInk8pNxxFwfvuwu/1Ur0JZPQxMTiLiul/ttv8FSZ5e+EcGaIYN3JqVVKfjc6kw++3cdzH23lvukDCdOJ/9auRt8jC3dZKaZhuaBQYDpvCLqUbuhSuhGeE1zeNPWBh1HHxsrjBQCS/u9P2H/5GU0b85Yr2uhWaR6xrs/MDHmIiBgx6sTzmpZO2t9ntjnCvHmAn2Zk6Pk0kZEYBw2WV5IzDR2GNiUFbXIykeMuwJQ7HEN2b2KnXYnk8+FvaEATf3hQ3ZEzqUUMH8mRoi+ehGXVShRqdXCO/IREkm+9nbqvvsS2LY/kW+9AqdPjra1Bn9mDwkceQPJ6Q1pM4qZeSXi//hiHDMW5P5+yF59Dm5RMzO+mUPn2v1HHxJA+82m5CVmflk73Rx7HU35IDtTyua64KviGQ89eaJOSUag1WDeuR5eWTuYNMwgQfIXRW10lt0ZoEhJDWl0AkCSMA84l5a57UWg0KBQKFBotsZdPQfL7MfTLQR0ZSXjOAHxNI+YNTTMdpt7/EL6Gevk7Ezt1GvXLl+E1m9v+zxV+M6LP+hS0d591M0mSWLqhmIUrCxg/OJXrL8o+Lec9kzpCn3VH8mvLI+B2g0Jx1MVPTpfgK0VrMQ0ZetxXlE6n+HgTlcVmnAfy0aWmoY6KavNh4lTVf/8duu7d0cTFo9TpUBmN+Brqsf28jchxF4S0XPmdzmCt/txBbbZoST4fVfM+JGr8RHQpKXjMlQBoE4+/xG1bpEAAV8FBwnr2kr8fnuoq3MVFGM7pi7usVH41sXLuO/JIeIVOR6/X3zrh63hra1BHRh11kFzRk38jPKc/8dOvaXP/sYg+65MngvUp6CjButlH3+XzQ14Z904fyICs47932ZGIYB1KlEcoUR6hjlceks+H88B+ahYtIGbSZRgHnXfaru23WlFotSf1sCaC9cnrvMOHhVauviCLlLhwZi34hWUbS9o7O4IgtJPmCYDSHnvitAZqCE56dCZbVYQgEazPIhq1ipsuCU7IsHDlQby+XzfXsyAIgtAxiWB9lumVGsVdV/Q+7HxgAAAdkklEQVTHH5DYfrD2+AcIgiAIHZ4I1meh3mlRKBTw+uIdrN5efvwDBEEQhA5NBOuzkEGv4YEZg8hKieCDb/OpqD351XYEQRCE9ieC9VmqT3o0t0/NQamAv72zkW83iQFngiAInZUI1mexmAg9f/v9EPplxLDgp4PYnN72zpIgCIJwEkSwPsulJhiZOqYHAUnitYXbeWPJTgLi1XpBEIRORQTrLiAj2URcpJ6D5Y1s2VvFDjFKXBAEoVMRwboLUCoUPHnzUGbdPZpok45vNhTj8weQJAmb04v1FNfVFgRBEH5bYsWHLiJcrwFg8oh0Plyez59fWkFyrIGKWgdpCUZm3jLsOGcQBEEQ2osI1l3M2IEpuDx+XB4/G3YFFxYoqbJRXmMnJe7XLyYvCIIg/PZEsO5i1Collw5PB+CKsT1osLm5//W1bNxtZtpYsT6tIAhCRySCdRcXZdRxTlo0X64rQqNWMmZAMpFGMUm/IAhCRyIGmAmMGZgMwKJVBcxasJ16q7udcyQIgiC0JIK1QG6fRF6/byx3TM2htMrGk3M3caja1t7ZEgRBEJqIZnABhUJBmE7NkHMSeCounJfnbePl+T8THx1GbISeXqmRdIsLp2dqJAD1jW7iosLaOdeCIAhdhwjWQohuceHcNqUfsz7bzoEyCwewsHG3GYBLhqXh9PhY+XM5L98xkpgIfTvnVhAEoWsQwVpopXdaNK/cMYpGh4ef99ewbX81+8ssLGuxGMinPx1gUm466UkmADxePxq1EoVC0V7ZFgRBOGuJYC20yaBXY9CruSQ3jUty0/D6Avy4tYwDZRYcbh+b9lSxaU8Vf7ysD15fgA+X5zNldCaNdg/9s2Lp3yO2vW9BEAThrKGQJLGqw8mqrbURCJxc8cXHm6iutp7mHJ05jQ4PbyzaQX6Zpc39cx+5EACHy8umvVXk9kkkTKfG7fGDAnQaVUj6zl4ep5soj1CiPEJ11vJQKhXExhrbOxudkqhZCyclwqDlT5f347mP8hg7IIWeqZG8PP9nef+9s1dzTno0Bw9ZqG10U1nr4JoLe/KP9zYD8NiNgwkEJEwGbXvdgiAIQqchgrVw0mIj9bx0+0i5n3pSbholZitKpZIdBbVs2lOFSqkgLlLPd1tKcbh9VNQ6ALhr1mrCdCr+MKkPQ89JaM/bEARB6PBEsBZOScsBZdMv6Cn/LEkSRZVW9FoVOo2KB95Yx5rtFaQnmXB7/FTWOTCFaXlzyU625yRRXGWjd2oUk4anUVplw+n2Mbh3Ai6PD51GRbXFRXKMAaVSwbb91bg8fkb0SzpqvvyBACqlmEZAEISzgwjWwm9CoVCQmRwhf777ygEADOwZi9Ptp6DcQu+0aJ5+bwvrdlaSmmjkh61l/LC1TD4mfVMpxebD/XLGMA03XtybN5fsBKB39yi+2VBCSpyBkTnJ6LTBfvBFqwpYv7OCmbcMwx+QUCkV8qpjDTY3Xl+AePGeuCAInYgYYHYKuvIAs9PF7vLidPs4Jyue/yzajtPjIzMpgm82llBWbWN0/2Q8Pj+b9lQd8zyRRi25fRIprrSyr7QhZF+UUcvEoWlU1tlZ9UsFAIN6xdG/Ryyj+ifh9QVwewNEm3R4fX7Kaxy4PD56pkaG1M59/gD/W7qHETlJ5GT+utHuv7amL74foUR5hOqs5SEGmJ08EaxPgQjWp8+R5eHx+qltdJEcG1y2MxCQWLqhmEWrCshMjmDG+J489+FWAP4w6Rw++XE/TrcfgJzMGBKiw9i0pwqb03vM6yoVCgJNvwIxETokCXlu9DCdmowkEx6vn4uGdqeowiq/a37ZyAymjcmUuwEOVdv4en0x08b2IDZCzy8Haqi2uHB5fJjrHGzdX8MVY3qQ2y+RiKZBdSVmK1Emnfy5WVm1jZjocAzqk39n3evzEwggtzacSZIknfb37cXvS6jOWh4iWJ88EaxPgQjWp8+JlIfH6+fVz7YzeUQ6fTNiKDFbsTm99M2IwerwYHV4SY41yIFCkiQ83gBb9lWxensFlwxLY+2OCvaW1DN5RAY6rYqCQxaKzVbKqu3ydRQKaP6tCNOpUCoU2F2+VvnpFh/OiH5JrNh2iBqLS96uVinx+QNt3kNyrAGDTk33RBOrfylHpVRw1flZHKqxEx8VxjcbiuVrDesTHHh3w8TerNtRgV+SQIIwvZohvRPw+QPklzZwTno0u4vqcHn85O2rxmJzY7EHy2NgVixTxmSSkRSBw+VjR0EtOq2K9EQT0SYdtRYXMRHBVdYa7R55xTWb08u+kgYG9Yojv7SBXUV1TB6Rjl4b7DnbfrCWcL2azOQINu+tol9mDAa9mre/2IW5zsmwvglMGNwdjVpJQJIoKG8kXK8mNkKP9ojX9mxOL8YwDSVmK1X1Tgb3jgeg2uIioam74sjvx/6yBry+AH0zYo75nTnyOkoFGJq6RI7F7vISplWjVCpotHswGTQdasKfzvr3QwTrkyeC9SkQwfr0OZPl0VbNr7LOwdodFazbWcntU3OIMmrZll/D+MGpBCSJ/aUNaLUqeiRHcKjazrYDNSzdUIzb4yczOYJBveLYtMcsB/0+6dHcPjUHZdN1dhbWsmR1IZV1DvmaOo0Kt9d/1Hy2rPUfSaVUoNOocLhbP0QYdGrCdCpqGw+vnqZRK/H6AiGfNSplq+NzMmOorHPQYPPg8wcYPSCZvcX11Fhc6LQquicYSY4xsHp7sDshMToMc72TKKOWbvFGdhXWyefK7ZuIUa9hZ1Ed5qb7PictigeuHYTD5WN3UR31Vjef/HiArG4RHDzUKKepa3RT1eAkKyWC0QOSmTiyB/sLa3B5/PRMjeSPL/wEQLhezegByYRp1fgCAaaN6cHekgbmfZ/PkN4J/PTzIZKiDfz+kt68vngnNqeX/5vch89WHOTS4enk9k1EkiR8/gAbdwcfOtbvquSzFQfJyYwhMcbAD3ll3HzpOQzpnUBRRSMur5/uCUYkKViOUUYdVfUODlXbObdXHAqFAn8gQI3FRYxJj0Yd2v2xaY+Z5ZtLSUswcs2FvUJaPwKShALwB4J5an448nj9KJUKdhbU0TstirTU6DZ/XworGlEpFcRHhRGma3tIUn5pA5V1DmIj9fRJi0apDOZ30+4qEmMM6DRKTOFaPF4/5jonMRE6kmPDkSSJ2kYXkhRcWvfI+zoRIlifPBGsT4EI1qdPZyyP8ho7JVVWhvVJRKlQUG918/OBGspr7EwYkkpitKHVMbsK6zhYbmFfSQNXX9ATq8NDmE5NTISeBpubxOgwGh1eSmocJEfp2bC7kuWbSrltSg590qNQKhWY65xs2mvGXOckI8nExt1mJg7tTmZyBOW1doaek4BCoSC/qe/++Y+2Em3SMW5gCqkJRuxOL5v3VrGzRWBtKS3BSLf4cHRaNSu2HQJgZE4Seq2Kg4caKTZb5eBvMmiQJOTuhj7p0VTU2mmweULOOWl4GrUWF5v2VJEaH47L4w9pjQBQAGMGpvDLgRpS4sLZU1x/qv9Frei0quDEPC0YwzTEmHSUVB17pbljPTw1S080oVYpqGl0YbF5iAjXkpFkIiBJqJVKymvsVDU4iTbpaLC6CdOp6ZsZQ05mDLsK69hZWIezxQNUdvcovL4AJWYrcVFh8kNPZkoEMUYdCgV4fAGKKq1Em3QUVwZ/h8L1aiYM6Y5KqaCi1s5lIzNQKRUsWlUQMv7j4mHdiTBoWbDiYMh9qJQK/C3+tmWnRmJz+SivCT6MThmdyZTRmccsizbLUATrkyaC9SkQwfr0EeURqmV5uD3+U+p7rqi1E2XUtapp1TW6cLp9FFVaGdQrrikASxj0h9OV19jZVVjH+YNS0KhVIfnJL20gLdGIUqFgd1E9kiTROy0KnVbF+p1mdhXVMW1sD+xOL5nJEUiSxOrtFSxaeZBGhxdDU6C6clwPTGFa3F4/0SadfG1znYOD5Rbio8KY98MBwvVqslIiKDHb8PoD3DAxm2ijjveW7aN3WhQbdlVi0GvI6hbBiH5JbNptpnuCEYfbT4nZioTE2AEprG9amGbJ6gJa/vXrkx4tPyDMvHkoa3ZUkNs3kV2FdSxZXSinsdg9xEbo2VFQCwS7Svr3iMUYpmHb/hrqrW7O7RlHn4xodjcF4EBAovlS489L5YpxPfh5fw1rdlTI14wM15KZHMHPB2pC/p8So8PQaVWUmG0hDwyJ0WEolQrqGt1yC01shJ7fjcpg1fZyuaWiJY1ayaTcNPRaNZ/+dEDerlUr8fhad90Mzo6nvNYuz48wun8yPVIiyO2beNSa+7GIYH3yRLA+BSJYnz6iPEKdzeXh9vhxeXxy//iJiI01UlNjPe39xiVmK3aXj7pGFyNzktiyr5q4SH3Ia4cer5/9hyxkJkWEPMhY7B7C9Wq5ORzA4fJRY3GSlmiS01XVOwhIwbcJ3F4/WSmRIXnYU1zPrsI6pozORKNWUlFrJzHagMPt45cDNeT2TUShgEPVdpJjDXy5rojc/il0iw725wckCaVCQaPDg1atRK9VEwhI1DS6MOjU7CupZ19JAwa9mtH9k+Xlbc11DtbvqqTO6uaykRlU1TtYtLKA6ednkRJvJFyvRq0K3tdri3aw/WAtr9839qSav5uJYH3yRLA+BSJYnz6iPEKJ8gglyiPUmS4Pnz+A1xc4qdp0SyJYn7wuN8XT7t27ue222xg2bBgDBgxg6tSpzJ8/H/HMIgiC0Da1SnnKgVo4NV2q9Lds2cLNN9+MJElMmjSJhIQEVq5cyZNPPsm+fft48skn2zuLgiAIgtBKlwnWPp+Pxx9/HI/Hw9tvv824ceMAuOeee7j55pv5+OOPmTx5MkOGDGnnnAqCIAhCqC7TDL5x40aKiorIzc2VAzWAVqvlnnvuAWD+/PntlT1BEARBOKouE6w3bdoEwOjRo1vtGzx4MAaDQU4jCIIgCB1JlwnWhYXB9yTT09Nb7VOpVKSmpmI2m3E6nWc6a4IgCIJwTF2mz9pmC85OZDKZ2twfHh5cMMJqtRIWdmLLJ57qKwjx8W3npasS5RFKlEcoUR6hRHl0LV0mWP8W6uvtJ/2edWyskdraY09v2JWI8gglyiOUKI9QnbU8lEoF0dHh7Z2NTqnLBGujMVgLtlrbnkjAbg/OeXu0mndbTvVLJyYHCCXKI5Qoj1CiPEKJ8uhaukyfdWZmcNL54uLiVvv8fj9lZWUkJiaecBO4IAiCIJwpXSZYDxs2DIA1a9a02peXl4fD4ZDTCIIgCEJH0mWCdW5uLhkZGWzcuJGVK1fK2z0eD6+++ioA11xzTXtlTxAEQRCOqkst5LF582ZuueUWJEni0ksvJT4+npUrV7J//36uvfZaZs6c2d5ZFARBEIRWulSwBti1axdz5swhLy8Pt9tNRkYG1157LTNmzDjty+8JgiAIwunQ5YK1IAiCIHQ2XabPWhAEQRA6KxGsBUEQBKGDE8FaEARBEDo4EawFQRAEoYPrMtONdgS7d+9m9uzZbN26FZfLRY8ePZgxYwbXXHNNpx6J/vnnn7N161b27NnDvn37cLlc/OUvf+Guu+5qM73X62Xu3LksWbKEsrIyTCYTI0aM4N5776V79+5tHrN+/Xr+/e9/s3PnTgKBAL179+amm25i0qRJv+WtnZT6+nq+//57VqxYQX5+PmazGY1GQ3Z2NldccQVXXnklSmXoc7IkScyfP59PPvmEgoIC9Ho9gwcP5u6776ZPnz5tXqczfZ9efPFFdu7cSVFREQ0NDRgMBrp168bll1/O1VdfjcFgCEl/tpfHkT7//HMeeughAJ577jmuuOKKVmm6WpkIoVQzxcvFZ8SWLVu44YYbKCkpYcKECYwcOZLCwkIWLlxIXV0d559/fntn8aTdcccdrF+/HpfLRWxsLBaLhWHDhpGbm9sqbSAQ4C9/+Qvz5s0jLi6OKVOmEBsby7Jly1iyZAnjx48nOjo65JilS5dyxx13UF9fz+TJkxk0aBDbt29nwYIFhIWFcd55552pWz0hS5Ys4YknnsBqtTJ48GDGjBlDcnIymzdvZtmyZRw4cIBLLrkk5I/lU089xWuvvYZOp2Pq1KmkpaXx008/8cknnzBs2DBSUlJCrtHZvk+33norsbGxnHvuuYwYMYL09HSKi4v54osv+PHHH5kyZQparVZOf7aXR0tms5nbbrsNlUqF1+tlwoQJbQbfrlQmQhsk4Tfn9XqliRMnStnZ2dKKFSvk7W63W7ruuuuk7OxsafPmze2Yw1Ozdu1aqbS0VJIkSVq4cKGUnZ0tzZ49u820ixcvlrKzs6Xrr79ecrvd8vYVK1ZI2dnZ0u9///uQ9BaLRRo6dKiUk5Mj7d69W97e2NgoTZw4UerXr59UXFz8G9zVyVu3bp30ww8/SH6/P2R7VVWVNG7cOCk7O1tatmyZvH3Tpk1Sdna2NHHiRKmxsVHevnv3biknJ0eaOHGi5PP55O2d8fvkcrna3P7ggw9K2dnZ0n//+195W1coj5ZuueUW6cILL5Sef/55KTs7W1q4cGGrNF2tTITWRJ/1GbBx40aKiorIzc1l3Lhx8natVss999wDwPz589sre6ds5MiRpKamnlDa5vu85557QmpS48aNY9iwYWzYsCFksZVly5ZhsVi47LLLQmobJpOJ2267Da/Xy8KFC0/TnZweI0aM4MILL2zV1B0fH8+MGTMA2LRpk7y9uUxuv/32kFXf+vTpw2WXXUZRUREbN26Ut3fG75NOp2tz+8UXXwyELrDTFcqj2ccff8zatWt59tlnW3UFtNSVykRomwjWZ0DzH+bRo0e32jd48GAMBkPIH++zldvt5pdffsFgMLTZdD1mzBggNJA1/wFqq+yat3WmslOrg8NEVCqVvK05/6NGjWqVvvkeW/4hPpu+TytWrACgd+/e8rauUh4lJSW89NJLXHfddQwfPvyYabtKmQhHJwaYnQGFhYUApKent9qnUqlITU0lPz8fp9N5Vi/RWVJSQiAQIDU1NSRYNWsun6KiInlb889tlV18fDwGg6HNZU87Ip/Px+effw4cfjBxOBxUVVVhMBiIj49vdUzzfbe8x878ffr3v/+N1+vFYrGwdetWdu3axciRI7nqqquArlMegUCAhx9+mJiYGB544IFjpu0qZSIcmwjWZ4DNZgMIab5qKTw8HACr1XpW/+JYrVYAjEZjm/ubtzeng+OXndFopL6+/nRm8zfzyiuvkJ+fz9ixY+Vg3Xyvx7q/lumgc3+f3nrrLRwOh/x56tSpPPHEE3KXSFcpj//+979s27aN995775jN39B1ykQ4NtEMLghnwPvvv8/cuXPJzMzkhRdeaO/stJtt27axd+9eVq9ezYsvvsi6deu46qqrKC8vb++snTH5+fnMnj2b66+/vs03JgShLSJYnwFtPfm2ZLfbgaM/BZ8tmu+v+an/SG3VBo5XdjabjYiIiNOZzdPuww8/5NlnnyUrK4sPPviAmJgYeV/zvR7r/lqmg87/fVIoFCQkJDBlyhRee+01CgoKeOaZZ4CuUR4PP/wwiYmJ3H///SeUviuUiXB8IlifAZmZmQBt9q36/X7KyspITEw865uj0tLSUCqVlJWV4ff7W+1vLp+MjAx5W/PPbZVddXU1DoejzX65juLdd9/l6aefJjs7mw8++KBVn6PBYCAhIQGHw0F1dXWr45vvu+U9nk3fp4EDBxIRESEPduoK5bF7925KS0sZNGgQvXv3lv+99tprADz66KP07t2bOXPmAF2jTITjE8H6DBg2bBgAa9asabUvLy8Ph8Mhpzmb6XQ6Bg4ciMPhYOvWra32r169GiCkLJqbCdsqu+ZtHbXs3n77bZ577jn69OnDe++9R2xsbJvpmvO/du3aVvua77Flc+nZ9H2y2+3YbLaQAYdne3lcddVVbf7r27cvAEOHDg35DGd/mQgnoL1f9O4KTmSCgk2bNrVjDk+f402KsmjRorN+UhRJkqTXXntNys7OlqZNmybV19cfM+3GjRtP+4QXHen7VFBQIFmt1lbbPR6P9Mgjj0jZ2dnSgw8+KG8/28vjaGbPnn3USVG6apkIhykkSZLa+4GhK9i8eTO33HILkiRx6aWXEh8fz8qVK9m/fz/XXnstnXnW1wULFpCXlwcEm922bt3KOeecI09iMnjwYKZPnw4EX1m5/fbbWbFiBb169WLcuHFUV1ezdOlSDAYD8+bNIysrK+T8X3/9Nffffz8Gg4HJkycTHh7O8uXLOXToEA888AB/+tOfzuwNH8fixYt55JFHUKlU3HDDDW32C/bp04cJEybIn2fOnMm8efPo1q0bEydOxG638/XXX+P1evnf//7HkCFDQo7vTN+nd999l3/9618MHjyY1NRUIiMjqa6uZt26dZjNZjIzM1t1EZzN5XE0c+bM4bXXXjvq3OBdsUyEw0SwPoN27drFnDlzyMvLw+12k5GRwbXXXsuMGTM69aT6jzzyCIsXLz7q/mnTpvH888/Lnz0eD3PnzmXx4sUcOnQIo9HIqFGjjruQx5tvvikv5JGdnc3NN9/cIRfyaP6jeyxHlokkScybN4/58+dTVFSETqeTF2lo2RzaUmf5PuXn5zNv3jzy8vIwm81YrVbCw8PJyspiwoQJXH/99a36Ts/m8jia4wXrrlgmwmEiWAuCIAhCBycGmAmCIAhCByeCtSAIgiB0cCJYC4IgCEIHJ4K1IAiCIHRwIlgLgiAIQgcngrUgCIIgdHAiWAuCIAhCByfWsxaEdrJo0SIeffTRNveZTCa2bNlyhnMU9Mgjj7Bu3TpWrVrVLtcXBKE1EawFoZ29+uqrJCUlhWxrubCFIAiCCNaC0M769OnToZf5FASh/Yk+a0HowBYtWkTv3r3ZvHkzd9xxB4MGDSI3N5ennnoKl8sVkraqqoqHHnqI3NxccnJyuPzyy/n8889bnbO0tJQHH3yQUaNGkZOTw/jx43nmmWdapdu9ezfXXXcdAwcOZOLEicybN+83u09BEI5N1KwFoZ35/X58Pl/INqVSiVJ5+Fn6wQcfZNKkSVx33XVs376dN954A6fTKS8G4nA4uPHGG7FYLPz1r38lKSmJL774goceegiXy8U111wDBAP19OnTCQsL4+677yY9PZ2KiopW6x7bbDbuv/9+brrpJu68804WLVrEzJkzyczMZPjw4b9xiQiCcCQRrAWhnbW1ctj555/PW2+9JX8eO3YsDz/8MACjR49GoVAwe/Zsbr31VjIzM1m0aBFFRUW8//775ObmAjBu3Dhqa2uZNWsWV111FSqVijlz5uB2u/n8889JTEyUzz9t2rSQ69vtdp588kk5MA8dOpQ1a9bw9ddfi2AtCO1ABGtBaGevv/56SOAEiIiICPl8ZECfPHkys2bNYvv27WRmZrJ582YSExPlQN3sd7/7HY8++igHDhygd+/erF27lvPPP7/V9Y4UFhYWEpS1Wi0ZGRmUl5efzC0KgnCKRLAWhHbWq1ev4w4wi4uLC/kcGxsLgNlsBsBisRAfH3/U4ywWCwANDQ2tRp635ciHBQgGbI/Hc9xjBUE4/cQAM0HoBGpqakI+19bWAsg15MjIyFZpWh4XGRkJQHR0tBzgBUHoPESwFoRO4Jtvvgn5/PXXX6NUKhk4cCAAw4YNo7Kykry8vJB0X331FbGxsfTs2ROAUaNG8dNPP1FVVXVmMi4IwmkhmsEFoZ3t2bOH+vr6VttzcnLkn1etWsULL7zA6NGj2b59O6+//jpTp04lIyMDCA4Qe//997nrrru47777SExM5Msvv2Tt2rX84x//kCdZueuuu1i5ciUzZszgtttuIy0tDbPZzOrVq3n55ZfPyP0KgvDriWAtCO3snnvuaXP7+vXr5Z9feukl5s6dy/z589FoNEyfPl0eHQ5gMBj44IMPeOmll3j55Zex2+1kZmby4osvMmXKFDldamoqn376KbNmzeKVV17B4XCQmJjI+PHjf7sbFAThlCkkSZLaOxOCILStef7w5cuXi1nOBKELE33WgiAIgtDBiWAtCIIgCB2caAYXBEEQhA5O1KwFQRAEoYMTwVoQBEEQOjgRrAVBEAShgxPBWhAEQRA6OBGsBUEQBKGDE8FaEARBEDq4/w+CD594dTImBAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('Loss Curve for Parallel is All You Want Model')\n",
    "plt.ylabel('Loss', fontsize=16)\n",
    "plt.xlabel('Epoch', fontsize=16)\n",
    "plt.plot(train_losses[:],'b')\n",
    "plt.plot(valid_losses[:],'r')\n",
    "plt.legend(['Training loss','Validation loss'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9K5xAwdFp7-O"
   },
   "source": [
    "Looks good. Past about 78% validation accuracy, the model begins to overfit on the training set. Note that I ran the final model longer than necessary (500 epochs/20 minutes, vs the 200 epochs/8 minutes it takes to train to convergence as trained on a K80 from Google Colab). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dOoycLWt60EA"
   },
   "source": [
    "# Load the Trained Model from Checkpoint for Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "7ZMZvY3Z60EA",
    "outputId": "a1793149-4005-47da-eb4d-16b08fb7424c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded model from /content/gdrive/My Drive/DL/models/checkpoints/parallel_all_you_wantFINAL-429.pkl\n"
     ]
    }
   ],
   "source": [
    "# pick load folder  \n",
    "load_folder = '/content/gdrive/My Drive/DL/models/checkpoints'  \n",
    "\n",
    "# pick the epoch to load\n",
    "epoch = '429'\n",
    "model_name = f'parallel_all_you_wantFINAL-{epoch}.pkl'\n",
    "\n",
    "# make full load path\n",
    "load_path = os.path.join(load_folder, model_name)\n",
    "\n",
    "## instantiate empty model and populate with params from binary \n",
    "model = parallel_all_you_want(len(emotions_dict))\n",
    "load_checkpoint(optimizer, model, load_path)\n",
    "\n",
    "print(f'Loaded model from {load_path}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "At3PtzXZ60ED"
   },
   "source": [
    "# Evaluate the Model on Hold-Out Test Set\n",
    "Fingers crossed for generalizability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "jMw_Eecr60ED",
    "outputId": "d9bec06e-2781-436d-def2-e4730b97d11b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy is 80.44%\n"
     ]
    }
   ],
   "source": [
    "# reinitialize validation function with model from chosen checkpoint\n",
    "validate = make_validate_fnc(model,criterion)\n",
    "\n",
    "# Convert 4D test feature set array to tensor and move to GPU\n",
    "X_test_tensor = torch.tensor(X_test,device=device).float()\n",
    "# Convert 4D test label set array to tensor and move to GPU\n",
    "y_test_tensor = torch.tensor(y_test,dtype=torch.long,device=device)\n",
    "\n",
    "# Get the model's performance metrics using the validation function we defined\n",
    "test_loss, test_acc, predicted_emotions = validate(X_test_tensor,y_test_tensor)\n",
    "\n",
    "print(f'Test accuracy is {test_acc:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GTqJD8oPr1VC"
   },
   "source": [
    "Not too shabby. Past epoch 200, test accuracy is between 75-80%; past epoch 400, test accuracy is between 77-80%; the final few epochs' test accuracy is between 78-80% - so there is a marginal amount of learning going on at the very end. \n",
    "\n",
    "For some reference points: **[A May 2020 paper](https://www.sciencedirect.com/science/article/abs/pii/S1746809420300501) claims to have achieved a \"new SOTA\" on RAVDESS speech audio with 71.61% accuracy on 8 emotions**, while [another May 2020 paper](https://ieeexplore.ieee.org/document/9122698) claims to have achieved the \"new SOTA\" with an 90% F1 score on 8 emotions. Both use CNN architectures. However, the paper reporting 90% F1 score is inflated by testing on training samples: [See the GitHub issue on this paper.](https://github.com/marcogdepinto/emotion-classification-from-audio-files/issues/11)\n",
    "\n",
    "From the author with a 90% score: _\"Also, the previous versions of this work used audio features extracted from the videos of the RAVDESS dataset. This particular part of the pipeline has been removed because it was shuffling very similar files in the training and test sets, boosting accuracy of the model as a consequence (overfitting).\"_ \n",
    "\n",
    "The author has augmented the RAVDESS speech audio dataset with speech extracted from video data from RAVDESS - so the **dataset with 90% accuracy contained duplicate samples _before_ being split into training and test sets, causing leakage of test samples into the training set.** For reference, I achieved a 97% accuracy with my model when I similarly augmented the dataset with duplicate speech audio before splitting data. **I corrected this data-leakage issue by augmenting data only _after_ splitting into train/validation/test sets.**\n",
    "\n",
    "This is an excellent example of \"if it's too good to be true\". More conservative estimates are more likely to be reproducible, in my opinion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oQ-hw8EX60EH"
   },
   "source": [
    "# Analyze Performance on Test Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 481
    },
    "id": "FigquBeQ60EI",
    "outputId": "ebf7848e-1297-43a7-831e-83bc3b500ba1"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+wAAAHQCAYAAAA7/uB3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVxN6R8H8E+bFq2oENlyW7RIG4mUJTtFZStjN3YZU3ZjG8tUiLGFMZZkSYmQLSSVUogI7ZbI0krr+f3R797purdNy73p+369er30nOfc8z3n5D7Pc86ziDAMw4AQQgghhBBCCCFCRVTQARBCCCGEEEIIIYQXNdgJIYQQQgghhBAhRA12QgghhBBCCCFECFGDnRBCCCGEEEIIEULUYCeEEEIIIYQQQoQQNdgJIYQQQgghhBAhRA128lNjGAaXL1/GokWLYG1tDQMDAxgYGKB///5YtGgRrly5gpKSEoHGeO/ePUyaNAk9evSApqYmNDU1kZ6e3iDHjoiIgKamJpycnBrkeDXFjk9TUxPa2tp4+/ZthXkTEhI4eTU1NZGQkNCAkVaOHRMhpOmytrbmfBfcv3+/wnzsPHl5eQ0YneC5ublBU1MTfn5+1UoXtPT0dGhqasLa2vqH9v/69SuOHDmCKVOmwMLCArq6uujRowdGjBiBtWvXIiYmpo4jrrnjx49jxIgR0NfXr9W5/ggvLy9oamrCy8urwY5ZE+z4NDU1YWNjU2neY8eOcfKamZk1UIRVE/Y6IPkPNdjJT+vdu3ewt7fHwoULcfnyZcjJycHS0hL9+vWDgoICLl++jAULFsDBwUFgMb59+xZz5sxBVFQUdHV1MXr0aNja2kJGRkZgMQmr0tJSnD9/vsLt586dq/NjCnuFgRDSOHl4eAg6BCJAkZGRGDhwIDZt2oT79+9DXV0dgwYNgpmZGb59+wYfHx+MGzcOW7ZsEViM165dw7p165CWlgYLCwvY2tpW2TBtqpKTkyt9wFIf9RMnJydoamoiIiKizj+bCB9xQQdASH349OkTxo8fjzdv3qBnz55Yu3YtOnXqxJUnIyMD+/btw8WLFwUUJXD37l3k5+dj1KhR2Lp1a4MfX19fH0FBQZCWlm7wY9dEp06dkJGRgXPnzmHWrFk820tKShAYGAhlZWWIiYnh3bt3AoiyYkFBQYIOgRAiJKSlpfHgwQOEhISgX79+gg5H6Lm4uGDGjBlQUVERdCh1IioqClOnTkVRURGcnJywYMECyMvLc+WJi4uDh4cHUlJSBBQlEBwcDABYuXIlxo4d2+DHnzhxIoYOHQolJaUGP3ZNdOvWDU+ePMG5c+dgaGjIs/3FixeIi4uDrq4u4uLiBBBhxRpLHZDQG3byk1q7di3evHkDExMTeHt78zTWAUBVVRWrV6/G7t27BRBhGXbDsn379gI5vrS0NLp06YK2bdsK5PjVJSMjAxsbGyQlJeHhw4c820NDQ/HhwweMGDECYmJiAoiwcl26dEGXLl0EHQYhRAhMnDgRALB9+3YwDCPgaISfiooKunTpAjk5OUGHUmuFhYVYsmQJioqKMGvWLKxcuZKnsQ4Aurq6OHjwIKZOnSqAKMsIun7SokULdOnSBS1atBDI8avL1NQUampquHTpEgoLC3m2s9+u29raNnRoVWosdUBCDXbyE0pOTuY8GV69ejUkJCQqzW9sbMyT9vHjR2zevBk2NjbQ09ODsbExJk6cCH9/f74VrPJj7F69eoX58+fDzMwMenp6sLW15XnD6ufnx9XVeteuXZzxTW5ublx52L9/r6Lu2iUlJfD398f48eM54+J69+4Ne3t7eHp6oqCggJO3qvFLUVFRmDNnDnr16gVdXV306dMHv/32W4Xjw8uP1fb394ednR0MDAxgamqKBQsWIDU1le9+1TF69GgA/LuWscc2svPw8+LFC2zfvh2Ojo5c12Xu3LmIjo7myW9tbY1du3YB4L4/319zdhrDMDhx4gTs7OxgaGjI9XfFbwz7smXLoKmpiQULFvAcu7i4GOPGjYOmpiZ27NhR2WUhhDQyI0aMQNeuXREfH49Lly7VaN/alE1PnjzhfJ9raWnh2rVrALi/n06fPo3Ro0fDwMAAFhYWWL9+PWcs/ZcvX7Bhwwb069cPenp6GDp0aIXjyh8+fIjNmzfDzs6OU35YWlpWWn5UhN8Y9vLjhyv64VeuJSQkwM3NDf369YOuri7MzMwwc+bMSrsVP3nyBLNmzYKxsTEMDQ3h4OBQ4/vGFhAQgHfv3kFFRQXz58+vNK+IiAjf+smzZ8+wZMkS9OnTB7q6ujA3N6+wHAP+mzshPT0dt27dwoQJE2BoaIgePXpg2rRpePLkCVd+9vVmXxNnZ2fONWXfg6rmFaiou3Z2djb+/vtvjBw5EiYmJtDX14elpSWmTp0KX19frryVDUkrLS2Fn58fJkyYAGNjY+jp6cHGxgZbtmzBp0+fePKXr+sUFhZi586dGDhwIHR1ddG3b19s3LgR+fn5fM+lKiIiIhg9ejSys7M5/6fYSkpKcP78ebRs2RJ9+vSp8DPu3r2LtWvXYsSIETA1NYWenh769++P1atX4/Xr11x52XMnREZGAuC+P+WveflzzsvLw7Zt2zjnPGfOHJ48bN++fcPw4cOhqamJo0eP8sT67t07mJmZQVtbG2FhYT90zUjNUZd48tMJCQkBwzDQ0tICi8Wq8f5JSUlwdnbG+/fv0bp1a/Tv3x+5ubmIiIhAVFQU7ty5g7/++gsiIiI8+z59+hTr169HmzZtYG5ujvT0dDx69AiLFy9GSUkJRowYAQBQV1eHra0t4uPj8ezZM2hpaUFbWxsAYGRkVKvzX7ZsGQICAiAtLQ0jIyMoKiri48ePSE5Oxt69ezFp0iQoKytX+TlHjx7Fxo0bwTAMDA0NoaamhpcvXyIwMBBXrlzB9u3b0b9/f777enh44NChQzA2NoalpSViY2Nx5coVPHjwAIGBgT/Uxc3MzIzzFHv58uVo1qwZACArKws3btyAjo5OpRO7/fPPPzh79iw0NDSgo6MDKSkpJCcn49q1a7h58ya2bduGYcOGcfLb2NggLCyM5/4A4Po32x9//IHTp0/DyMgIVlZWePPmTaXns2rVKs51OXnyJMaNG8fZtnPnTsTExMDY2Bjz5s2r9jUihAg/UVFRLFy4EPPmzcOOHTtgY2NTrZ5BtSmboqOjsXr1aqipqaFXr174/PkzxMW5q4BbtmzBsWPHYGZmhrZt2yI6OhrHjh1DYmIiPDw8MG7cOHz9+hU9evRAZmYmoqKisGzZMoiKivI8LPX09MT9+/ehoaGB7t27Q0xMDC9evEBgYCCuXr0Kb29vmJiY/PA11NbWrvCNZUxMDJKTkyEqyv1OKiAgACtWrEBRURG0tLSgr6+P9+/fIzQ0FLdv38aaNWswfvx4rn3u3buHmTNnorCwECwWC127dsXr16+xaNEiODs71zjuGzduAAAGDx5c5csEfoKDg+Hi4sI5B1NTU6Snp+PatWu4ceMGVq1ahQkTJvDd19fXF97e3jA0NETfvn3x9OlThIaG4sGDB/Dz8+P0RGTXQe7cuYPMzExYWFhw6gzq6uo1jpktPz8fjo6OSExMhLKyMoyNjSElJYWMjAw8fvwYb968gaOjY5WfwzAMXFxccOnSJTRr1gxmZmaQlZXFgwcPcOjQIVy6dAlHjhxBhw4dePYtKirCtGnT8OzZM5iYmKBTp06IiorCv//+i8TERBw8ePCHzs3W1hZ///03AgICMHToUE46u/ffL7/8Uun/8bVr1yIjIwMaGhowMzNDUVERnj9/Dl9fX1y5cgU+Pj7o3LkzgLIeh7a2tnzvDwC0atWK67O/ffsGJycnpKSkwMTEBNra2lBUVKwwFikpKXh6emLs2LHYunUrTExMoKWlBaDsAcSSJUvw5csXzJo1C+bm5j90vcgPYAj5yfz2228Mi8Vili9f/kP729nZMSwWi3F1dWUKCgo46a9evWIsLCwYFovFHD9+nGsfV1dXhsViMSwWi9m/fz/XNm9vb4bFYjHW1tY8x9q5cyfDYrGYnTt38mw7e/YsJw5++O2bnp7OsFgspl+/fszHjx959omOjmby8/M5v4eHhzMsFouZNGkSV76nT58y2traTLdu3ZgbN25wbTt69CjDYrGYHj16MB8+fODaxr4GPXv2ZJ4/f85Jz83NZezt7RkWi8V4eXnxPR9+2PHZ2toyDMMw27dvZ1gsFnP58mVOnuPHjzMsFos5cuQIwzAMY2VlxbBYLK7jMwzDREREMOnp6TzHCAkJYbp168aYmJhwXRuGqfz+fH/OJiYmzJMnTyrN8734+HhGT0+P0dfXZxISEhiGYZiwsDBGS0uLMTU1Zd68eVPhcQkhjcv3301jxoxhWCwWc+rUKa587O+L3NxcrvTalk1eXl5MaWkpT1zs7b1792YSExM56e/evWN69uzJsFgsZtiwYczixYu5juvj41Nh2Xbr1i0mMzOTJ/3UqVMMi8ViBg8ezBMLO9azZ89WK52fmJgYRk9Pj9HT02NiYmI46U+fPmW6devGGBkZMWFhYTz7GBsbM926dWNevXrFSc/Pz2d69+7NsFgsZs+ePVz7BAUFMVpaWgyLxWKsrKyqjIutb9++DIvFYvz9/au9D1tGRgZjaGjIsFgsxsfHh2tbcHAwo62tzejo6DDx8fFc29h/d3p6ekxERAQnvbCwkJkzZw7DYrEYNzc3nuNNmjSJYbFYTHh4OM+2qu4Jv339/PwYFovFzJo1iykqKuLKX1BQwERGRnKlVVT+susglpaWTHJyMtdnuLi4MCwWixk7dizXPuy6BIvFYhwdHZkvX75wtqWkpDBGRkYMi8XiiaEy7Pg2b97MMAzDTJgwgdHR0eGqFy1cuJBhsVhMfHw8k5aWxrBYLMbU1JTns65evcpkZ2dzpRUXFzM7duxgWCwWM3XqVJ59Krs/35+zra0t3zphRXVAhmGYkydPcv6vsutG7DqYo6Mjzz0k9Yu6xJOfzufPnwHgh8Y93b9/H3FxcVBUVMTKlSs5b3EBoHPnzli8eDEA4PDhw3z37969O2bMmMGV5uzsDAUFBaSnp/N0baprHz9+BFD29oHf+ffo0aNak4v8+++/KCkpwejRo2FlZcW1bdKkSTA1NUVubi5OnTrFd/8FCxZw9W5o3rw5ZyxebWY0Zb9RKd8t3t/fHxISEhg+fHil+7LHmX3P0tISNjY2yMrKqlVs06dPh46OTo320dLSgqurK759+4bFixfj9evXWLp0KUpLS7Fx40a0adPmh+MhhAg3FxcXAMDff//Nd+xrebUtm7p06YI5c+bwffvOtnDhQq75XlRVVTm9wt6+fYtVq1ZxHdfe3h6Kiop8y7a+ffuiZcuWPMewt7eHoaEhEhMT8fLly0rPuabS0tIwZ84cFBYWYuvWrejevTtn2969e1FUVIRly5ahV69eXPt1794dc+bMQVFREVe37CtXruDDhw/o2rUrz2SnQ4YMwYABA2ocY23qJ6dOnUJeXh7Mzc25emQBwMCBAzFy5EgUFxfz7cYMlNVFTE1NOb9LSEhg9uzZAGpXLlcXu37Ss2dPnt4dzZo1q3aPC/bfuIuLC9db9GbNmmHNmjWQk5PDo0ePEBUVxbOvqKgoNmzYAAUFBU6auro6Ro4cCaD29ZPi4mIEBgYCKOv+f+PGDWhra3PeUFdkwIABPHM0iImJYcGCBVBVVUVYWBhyc3N/OLbVq1fX+G/O0dERQ4YMQWJiItavX4+IiAjs3bsX8vLycHd357mHpH7R1SakHPbauAMGDICsrCzP9pEjR2L16tVITU1FRkYGVFVVubbzG6MkISGBdu3aISsrC+/fv+fbaKwrnTt3hoyMDG7duoX9+/djxIgRP9ToYxd0FY0JHzNmDCIjIytcS7hv3748aeyK4Pv372scD5u6ujqMjIxw584dfPr0CZ8/f8bDhw/Rv3//ahVGOTk5uHnzJp49e4bs7GwUFxcDKBvfDpTNf/CjKhoeUJWJEyciPDwcwcHBGDlyJHJzczFp0qQfqgwSQhoPc3NzmJmZISIiAj4+Ppg8eXKFeWtbNllbW/N0Ef9e7969edLYXaC7devGM5RJTEwMampq+PLlC9+y7ePHj7hx4wZevXqFnJwclJSUAAAyMzMBlH3fdu3atdKYqis7OxuzZs3Cx48fsWTJEgwePJizrbS0FKGhoRATE8OgQYP47s8eK15+UlP2GOFhw4bxfdAxatQoznw5DaE65fK5c+cavFyuLl1dXQCAt7c3WrZsiX79+tV4IsF3794hPT0dEhISXEPY2OTl5TFw4ED4+fnh/v37PHMAtGnTBhoaGjz71cV1GDx4MDZs2IBz585hypQpuHjxIgoKCiqdW6e8169fIyQkBMnJycjLy0NpaSmAsjltSktLkZqaWuOXAkBZF/nyD69qYv369Xj8+DHOnj2LK1euoLS0FOvXr6/Xeizhjxrs5KfDrlTwm3ikKhkZGQCAdu3a8d0uLi6ONm3aVFgpat26Nd/9mjdvDgBVvkWpLVlZWWzevBkrV66Eu7s73N3d0aZNGxgZGaF///4YNGhQtZ6Ksq9DRbPDsq8PO9/3+D0kYF+DoqKiap1LRWxtbREdHY3AwEBOxa86s69evXoVy5cvR3Z2doV5avMEuzYF2IYNGxAeHo7s7GxoaGjA1dX1hz+LENJ4uLi4wNHREfv27cPYsWM535Pfq23ZVJ1ZoPmVXzIyMhVuAyou244fP44tW7ZwTXL6vdp835ZXVFSE+fPn49WrV7C3t8fMmTO5tn/58oVzLH6TuJVXvt7AvuYVfbf/yHe+kpIS3r17Vy/1E3Z5XZNymf3wp7blcnX07NkTs2bNgre3N3777TeIioqiS5cuMDExwdChQ6v1hp19bm3btq1wTHhl16GiFxh1UUeTlZXFwIEDcf78eTx79gz+/v4QFxfn9FKpjKenJw4cOMB5qMXPj/5/qc0M8HJycti0aROcnZ2Rm5uLsWPHcj0MIw2HGuzkp9OtWzecP39eIOtdVvUGoy6xn75+z8bGBr169UJISAjCwsIQFRWFCxcu4MKFC2CxWDhx4kS9L49Tn9dhyJAh2LhxI/z8/PD582coKirC0tKy0n3evn2LJUuWoKCgALNnz8bw4cOhpqYGaWlpiIiIwMPDA/v27avVEktSUlI/vO/t27c5DxIyMjKQkZEhsKV0CCENp3v37rCyssLNmzdx5MgRzuzNda0630+VfW/X5Dv90aNHWL9+PcTFxbFs2TL069cPrVu35sSwZMkSXLhwoc6WtFuzZg3Cw8Nhbm6OtWvX8mxnN4KqM3Sqvtf81tHRwbt37xAXF4dRo0bV67G+V9lwiLpWUf3ExcUFDg4OuHnzJsLDwxEdHY0TJ07gxIkTGD16NLZs2VKvcdV3Hc3W1hbnz5+Hp6cnYmNjYWVlxXdoSHmXL1/G3r17ISsri+XLl8PMzAwqKiqc4Sfjxo1DTEzMD/9/qU3dBAAuXLjA+Xd8fDwKCwu5hsaQhkENdvLTsbS0xObNm/Hs2TO8ePGiRl3u2G8l0tLS+G4vLi7G27dvufLWF/YMshUtNcJeI5UfeXl5jBw5kjMu6+XLl3B1dUVcXBz279+PJUuWVHpsVVVVpKamIi0tje95pqenc/I1NFlZWQwYMIAzTszJyanKwiMkJAQFBQWwsbHhjPUsLyUlpV5irY7U1FSsWbMG4uLiGDJkCAIDA7FkyRKcOHGCxogR0gQsWrQIISEhOHToEGeN9u8JU9lUleDgYDAMAycnJ/zyyy882+vy+3bfvn04e/Ysunbtip07d/L9zlRSUoKkpCSKi4uxbt26ajc22NexohU/fmROGisrK9y4cQOXL1/G77//XqOZ4lVVVZGYmIi0tDS+q8mw/zYa4v5XVT9h/y3y065dOzg5OcHJyQkMw+Du3btwcXGBv78/hg8fXunyZ+XvSUlJCd+37A15Hb7Xs2dPtGnTBiEhIQAAOzu7Kve5cuUKAGDx4sUYM2YMz3ZB1k+CgoJw6tQptG3bFurq6ggPD4eHh0eFyw2T+kOTzpGfTqdOnTBw4EAAwLp166rs6lV+YhJ2l6zr16/z7X4UGBiIoqIiqKur13thoKKiAqBsKZ/vFRYW1mhyFA0NDU7F6fnz51XmZ3cb9Pf357v97NmzAFCrZXlqY8yYMVBUVISiomK1xodlZWUB4N+t89OnTxWuJcqulLDHute1oqIiLF68GHl5eVi4cCG2bt0KMzMzPHz4ENu3b6+XYxJChIuWlhaGDRuGnJwcHDhwgG8eYSqbqlLZ9+2rV68QHx9fJ8cJCgqCp6cnWrVqhX379lXYc0xcXBzm5uYoKSnhWSe7MuxrfvHiRb5vN8+fP1/jmEeNGgUVFRW8f/+e7/ri3yu/tjq7XA4ICOCbl70mekOUy5XVT169elVpg708ERERWFhYwMbGBkDV9ZPWrVujXbt2KCoqwsWLF3m25+TkcO6xIOonoqKinMkY27dvj379+lW5T2X/X+7du1fh8Al2/aSybvS1kZ6ejtWrV0NMTAx//fUX3N3d0bJlS/zzzz+4detWvRyTVIwa7OSntHbtWrRu3RqRkZGYMWMG38nEMjMzsXHjRsydO5eTZmJigm7duuHLly/YuHEjV2M/OTkZnp6eAIApU6bU+zno6elBRkYGCQkJnCewQFljfdOmTXyf7j99+hRBQUE84wYZhuF8wVZnEjpnZ2eIiYnB39+f54v5xIkTiIyMhKysLOzt7X/k1GqtV69eiIiIQEREBGcim8qw1y8NDg7mjHsHyt4OrFy5ssJx7eyKb2JiYh1EzcvDwwNxcXEwNzfHjBkzICoqim3btkFJSQne3t64e/duvRyXECJcFixYAHFxcRw7dozvdmEqm6rC/r4NCAhAXl4eJ/3Tp09Yvnx5nTwAjY2NhZubGyQlJbFnz54qx5PPmTMH4uLi+OOPP/g22ktKShAeHo7Y2FhOmo2NDZSVlZGQkMDzICU4OBhXr16tcdySkpJwd3eHhIQE9u3bhw0bNvAtf549e4bp06dzrQvu4OAAGRkZhIWF8azQcv36dZw/fx7i4uJwcnKqcVw1ZWZmBqDsHqempnLSMzMzsWLFCr5d4q9evYqoqCiehx85OTmcBxPVGW/Nfvng4eHBdezCwkKsW7cO2dnZ0NfXr3K+gvoyd+5cRERE4Nq1a9XqzcH+/3L69Gmu/9fp6el8h3iwsesnr169ql3AfBQXF2PJkiXIycnB3LlzYWRkhFatWmHz5s0AADc3twaZqJD8h/pbkp9Sy5Yt4ePjg3nz5uHevXsYPHgwtLS0oK6uDlFRUaSnp+PJkycoLS2FgYEB177u7u5wdnaGn58f7t27B0NDQ+Tm5iI8PByFhYUYPnw4xo8fX+/nICMjg5kzZ2L79u1YtGgRjI2NIS8vj7i4OBQXF8POzo7zRJ3tzZs3WLx4MWRkZNCtWzeoqqqioKAAcXFxePv2LVq1aoXp06dXeWxtbW0sW7YMGzduxMyZM2FoaAg1NTXO25FmzZphy5YtUFZWrq/Tr1NWVlbQ0tLCs2fPYGNjA1NTU4iLi+P+/fsQFRXley0BwMLCAtLS0ggODsbEiRM5fz/W1tY/PCs82+3bt3H48GG0aNECW7du5YwvVFVVxZ9//onZs2fD1dUVAQEBVY6BI4Q0bh06dICdnV2FS2UCwlM2VcXOzg7//PMPnjx5goEDB8LIyAhFRUWIjIyEiooKBgwYUKM33fx4enqioKAAnTt35oyB/l7nzp05E9Dp6+vjzz//xIoVKzB37lyoq6ujc+fOkJWVRWZmJuLj45GVlYW1a9dyZtSWkZHBli1bMHv2bLi7uyMwMBAsFguvX79GTEwMnJycKlxCrTKmpqbw9vaGi4sLjh49Cl9fXxgYGEBVVRXfvn3Dy5cvOS8ZypfXKioq2LJlC1xcXLBq1Sr4+Pigc+fOnHhERESwevXqKpcQqwsmJibo3bs37t69i9GjR8PExATFxcV4+PAhtLS0YGhoiJiYGK59IiMj8e+//6Jly5bQ0dGBoqIisrOzER0djdzcXPTo0YPTO7IyEydORFRUFC5fvozhw4fDzMwMsrKyePDgAd69e4fWrVvjr7/+qq9Tr3NOTk44d+4cQkJCYGNjAz09PeTm5uL+/fvQ19eHkpISz7UEylaM8PPzw9atW3H37l1OPWHatGmchwA/aseOHYiNjYWpqSl+/fVXTnrfvn0xZcoUHDp0CL///jsOHTrUoHM3NWXUYCc/rbZt2+LMmTO4fPkyLl++jEePHnHelCorK2Pw4MEYNmwYrK2tufbr1KkT/P39sX//fty8eRNXr15Fs2bNoKenB3t7e4wePbrBJm/59ddfIS8vj+PHjyMmJgZycnKwsLDAkiVLcPr0aZ78BgYGcHFxQWRkJJKSkvDo0SNISUmhTZs2sLW1xaRJk6rd+HNycoKWlhYOHTqEmJgYzhrAw4cPx8yZM6GpqVnXp1tvJCQkcPz4cXh5eeHmzZsIDQ2FkpISrK2tsXDhwgorycrKyti7dy92796N+Ph4REdHg2EYtG7dulYN9g8fPnDGgG3evJnnwYeVlRWnMvj777/D29u7QScMIoQ0vLlz5yIgIKDCmdWFqWyqjIKCAs6cOQNPT0+Eh4cjJCQEysrKGDt2LObNm4dNmzbV+hjsN7iJiYkV9oAyNTXlmjF+5MiR0NfXx5EjRxAWFobw8HCIiopCWVkZPXr0gLW1Nc+yb71798aJEyfg5eWF6OhopKeno0uXLnB3d0f37t1/qMEOlI11vnr1Kk6fPo2QkBAkJCQgNjaWswzshAkTYGtrC319fa79Bg0ahDNnzuDAgQOIiIjAixcvICsri/79+2Pq1KkN9lZZREQEu3btwvbt23H58mXcvXsXKioqmDhxIubMmcP3xYCdnR2aNWuG6OhoPHv2DF++fIGioiJYLBZGjx4NW1vbao3pFxUVhaenJ/r27YszZ84gOjoahYWFaNu2LaZOnYoZM2b80Dr3gtKhQwf4+fnB3d0dMTExuHnzJtq2bYsZM2Zg1qxZmDZtGt/9+vfvjzVr1oId7w0AACAASURBVMDX1xf37t3Dt2/fAJT9ndemwR4WFoYDBw5AUVER27Zt42mQs+uY9+7dw/79+zF79uwfPhapPhGmrqbpJIQQQgghhBBCSJ2hfgyEEEIIIYQQQogQogY7IYQQQgghhBAihKjBTgghhBBCCCGECCFqsBNCCCGEEEIIIUKIGuyEEEIIIYQQQogQomXdGqGvN70FHUKlVIfXfsmW+pRfxH/JHGGiJC0r6BAavc9fcwUdAmkAxYWv6/wzizL5LxNVXRKtarcGLmmcqGyuHSqbmwYqm5sGKpvrFr1hJ4QQQgghhBBChBC9YSeEEELKKy0RdASEEEIIKa8Jl83UYCeEEELKY0oFHQEhhBBCymvCZTM12AkhhJDySptupYAQQggRSk24bKYx7IQQQgghhBBCiBCiN+yEEEJIOUwT7nZHCCGECKOmXDZTg50QQggprwl3uyOEEEKEUhMum6nBTgghhJTXhJ/iE0IIIUKpCZfNNIadEEIIIYQQQggRQvSGnRBCCCmvCa/1SgghhAilJlw2U4OdEEIIKa8Jd7sjhBBChFITLpupwU4IIYSU14QntiGEEEKEUhMum5vUGHY3NzdoamoiPT1d0KHAy8sLmpqaiIiIqNfjdJ+9je9Pr4XbOXkYhsHFiCdw9Q7EiFUH0HO+JwYv24uFf/vhcdKbeo2PHw2NTlixchGu3zyLxOT7eP3uEULvXcBvS+dARka6weOpiIiICBYumIG4x7eQm/0KSa/uY9uW1UIRYxeNTthz4C/cjQzCq9QopLyNRdj9S1i30Q2qqsqCDg9A44hRmO8xG8VY9ximtFY/pGaaYtlMCCGkZppy2Uxv2JuAHhrtMKaPPleauJgY59+FxSVYcTgImu1VYGOsBbVWCsjMysPp27Fw3nocG34ZimFm3Ros3knO9pgxcxIuBV3HKd8AFBUVo0/fnli99jfYjhmG/v3s8O1bQYPFUxH3v9ZiwfzpOOcfBE/PfdDW6op586aie3ddDBrsCIZhBBZbWzVVqLZWRtCFq3jzJgPFxcXQ0WHB+RcH2I4ZBiuLUcjM/CSw+BpLjMJ8jylGQgghhJCfX5NqsLu4uGDGjBlQVVUVdCgNSq2VQqUNbjFRUXi7jIMxqz1Xup2FPsb8cRjuZ0IwxEQHoqIi9R0qACDg3CV4/LUH2dk5nLRDB08g8VUylrrOg/NkB+zfd7RBYqmIjg4L8+ZOhd+5i3BwnMlJT0pOxY7tG+DoOAonT/oLLL47t8Jx51Y4T/q9sCgcPLID4ybaYdcObwFE9h9hj1HY7zFAMdabJtztThCaQtlcUlqKHeduI/BeHAqKitFLpyNWThwEJVkZvvmPBEfi9O1YfMrJR0v55pjU3xiO/QwBAA9epGPurjNc+QuLitG5TUucXjWl2jGJiorij/W/Y+LEMZCUksSN66FYuGAFPn38zDf/gIF9sXHTcnTspI6kpBQsd9uIG9dDAQBKSgo4cXIfWJqdISkpiY+Zn3Ds6Bls27qbs//uPVtgZd0b8vJyyM//iqvBt7Bi2UZ8+ZJd7Zhrcm5/blwOZ2cHSElJ4uq1W/h1jis+VnBudXncVX/8hnETbCElKYmQm6FYsnANPn3if1zr/n3wx0ZXdOjYHslJqVi9YjNCbtzlbP+Q9Rz5+V/BlPtO0tPui5zs3J86xpqciyDuc3UJe3xA44iRowmXzU2qS7yKigq6dOkCCQkJQYfS4IqKS5D/rZDvNnExUZ7GOgC0lG8OI1Y7fMrJx6ecvPoOkSMm5jFXY53t7NmLAABtHc0Gi6Ui4xxHQ1RUFDt3cjcovQ+eQF5ePiaOtxNQZJVLS30NAFBUlBdwJBUTlhgbwz2mGOsJU1q7H1IjTaFsPnQ5AiEPX+Ko6yRc2fwrAGDl4SC+eUMevsSeC2HYNHU4wnYswoYpQ+Hpdwv3niYDAHp0bYd7OxZxfu56LoSKohyGmurUKCaXJbMxbNhAWPezgzarNwDggLc737wdO7bHsRN74OG+B+3aGMDjr7047rMX6upqAIC8vK9YvGgVNDXM0a6NAUYOd4KD4yj8MmUc5zN2eR2EseFAtGtjAJMeAyEjLQV3z3U1irm6XH+fhxEjbWBuMRwdOhkDAI4c3lkvxypvoctMDBlqjcH97aGv0xcA8Pf+rXzzdujYDoePeWGHx350aW+MHR778c+xXWj//2vK5mA7DR3VenB+atsQbgwxVpeg7nN1CXt8QOOIkaMJl83VarBfu3YNzs7O6N27N3R1ddGnTx84OTnBx8eHk0dTUxNOTk5893dycoKmJncjq/w4MX9/f4wZMwbdu3fHqFGjAAB+fn7Q1NSEn58frl27Bnt7exgYGMDMzAy///47MjIyeI5jbW0Na2tr5OTkYOPGjbCysoKOjg7++ecfABWPk6vO+bF9/vwZW7duhY2NDfT09GBiYoKZM2ciNjaW77m/evUKc+bMgbGxMQwNDeHs7IwHDx5UfLHrwbWYBPRc4AnzRTtgtXQ3Np+8hpyv1etSnvE5FxLiYpCTkarnKKumptYaAPDhfaaAIwGMjQxQUlKCyPvc972goAAPHz6BsXF3AUXGTVKyGVq0UEKbtqroZ90b7jvKKkfXgm8JOLL/CGuMjeEeU4z1pLSkdj8NhMrm/wh72ewX+ghTbEzRTlkRctKSWGRnibtPkvDmYxZP3rQPn8Fqpwz9zm0BAAad1cBSa4WE9Pd8Pzs0LhGZ2XkYZa5Xo5h+mToenh77kJychuzsHKxeuRkDB/VD+/ZtefJOmDgGsTFx8D0ZgKKiIpzyDcDD2CeYMHEMAKCwsBDP4l+guLiYs09paSm6sjpzfo9/moD8/K//bWcYaHTtVKOYq2v6tInYtm03kpJSkZ2dA7dlGzF4sDXnAUN9cZrsAK/t3khJTkdOdi7+WL0N/Qf2RTs+19RxvC0exT7BmVPnUVRUhLOnA/H44VM4jh/d5GOsLkHd5+oS9viAxhEjRyMpm+tDlV3ifXx8sHbtWigrK8Pa2hpKSkr4+PEjnj17Bn9/f4wfP75WARw6dAj37t2DtbU1evXqhaKiIq7twcHBCA0NhY2NDXr16oXY2FgEBAQgKioKZ86cQYsWLbjyFxYWYvLkycjOzkafPn0gLS2N1q1b18n5paWlwdnZGW/evEHPnj1hZWWFrKwsXL9+HZMmTcLOnTthbW3Nyf/y5UuMHz8eOTk5sLKyQteuXZGQkABnZ2f07NmzVtetunQ7tsHAHiy0V1FC3rcChMYl4WRIDKJfpOHI0omQkWpW4b53HiciLvkthpvpQFJCsKMnREVF8bvrvLKKwqkAgcYCAG3aqiIz8xMKC3l7Lbx+8w7m5iaQkJDg+XtuaJOc7bH5r9Wc31NS0jF7+m8IvxctwKi4CWuMjeEeU4z1pBE8iaeyufGUzdn53/D2Uza01f/r8t9eWQmyUs2QkP4BbVsqcOW3MdaGf1gcYl6mw6CzGmJfvUbK+8/o3Y1/4/b0nVj0N2ShhRz/7vX8KCjIQV1dDbGxjzlpSUmpyMrKga6eNtLSuCec1dXTQmxsHFfaw4dx0NXT5ko7dcYblv3MIS0thbS0Nzh8kPvhyuIls/Hb0jmQk5NFfv5XzJjmUu2Yq0tBQR4dOrTDg5j/zi0xMQVZWdnQ19dB6v97cdU1eQU5tFdXw8Ny1yk5KQ3ZWTnopquF9O+uaTddLTyMfcKV9ujhU3TT0+JKO3hkO8QlJJCclAqv7QdwMfDqTx1jdQnqPleXsMcHNI4YuTSCsrm+VNkKO336NCQkJBAQEICWLVtybfv0qfYTQkVERMDX1xfa2tp8t9+8eRPe3t7o06cPJ83d3R379++Hh4cHNmzYwJX/w4cP6Nq1K44fPw5p6apnIK7J+bm6uuLdu3fYvXs3BgwYwEl///497O3tsWrVKvTu3RuSkpIAgHXr1iE7Oxvr1q2Do6MjJ//x48exbl39dAP73jG3SVy/j+ipi65qytgVcAfHb0RjxtBefPdLyfiMlf9chIqiLFzGWjVEqJXasnUVzHoaYe2abXj5IknQ4UBGWhoFBfyHGLAnxJORkUZWlmAbIUEXr+HFi0Q0by4DPX0dDB5qjZYtlQQa0/eENcbGcI8pxqaLyub/CHvZzB6OJictyZUuJyOFPD4TqLaQk8EAQxZmePpyJmRcam8NDTXe1TPefsrG3bgk7F/syLOtMrKysgCA7Czu4WdZWdmQl5flzS8ny5v3Sw60tLnzOoydDlFRURgZ6WPw0P74+JH7Xnm674Wn+1506NAOTs72SExMqVHc1SEnVxZTVhb32PgvX7IhLy9X58djk5VtDgDI/q47eFZWNicmrvxyzXmG/2VlZUNTW4Pzu93IyYgML+v5MWRYf+w58Bd+mTgPN67f+WljrC5B3efqEvb4gMYRIylTrS7x4uLiEBfnbdt//wT9Rzg4OFRYIQAAc3NzrgoBAMyePRvy8vK4cOEC37cyrq6u1aoQsFXn/OLj4xEdHY2hQ4dyVQiAsvF306ZNQ2ZmJu7duwcAePv2LSIiItCxY0fY29tz5R8/fjw6dOhQ7fjq2uRBJpAQF8OduES+219nfsGs7b4QgQh2zx9bo6f29WHlqsWY9etkHDp4Ah5/7RFoLGz5X79CUpJ/7wQpqbJKWfmuf4Ly9k0Gbofcw6WL17H1Ty/Mm+2G1euWYqHLzKp3biDCGmNjuMcUYz0pLa3dTwOhsrlxlM3snmzfD0XLyf+G5lKSPPn3B4Xh0v14+K6YjKjdS+C78hccux6Fc3cf8eQ9F/oIHVVb8J2HpjK5uWUNNnkF7kq5goI8T2MOAHJzcnnzKsrxHatcWlqK+/djkZ2dU+EY9ZSUdFwKuo4zfgchIlK3E9rm5JTFpKDAPQ+KoqI83/lx6kpubtlcP98/8FBQkOfExJU/J4+nUfR93ju3wlFQUIiCgkL4+13CmVPnMcZhxE8dY3UJ6j5Xl7DHBzSOGLk0krK5PlTZYB8xYgS+fv2KYcOGYdOmTbh27VqdPL1n09fXr3S7kZERT1rz5s2hpaWFr1+/IimJ+22rpKQkz5i8ylT3/GJiYgAAX758gZeXF8/Pw4cPAQCJiWWN4Pj4eABAjx49ICrKfZnLnj7znldDkRATg7KCLL7k8laSX2dmYbqHL/ILirB3oT268nmi35CWLV+I393m4+i/p7FowUqBxlLe2zcZaNWqBZo1422IqLVtjQ8fPgpXF9//e/rkOR4/eoop0yYIOpQKCUuMjeEeU4z1pBFMbENlc5nGUDbLy0ihTQt5PEv9bwx6+ocvyP1WyLeMjU/NgHX3rujSthVERESg0bYVrLp3xa1Hr7jyFZeU4tzdxxjTx6DGMWVl5SA19TUMuuty0jp2bA8FBTk8iXvGkz/u8TMYGHCvNqOv3w1xj+MrPIa4mDg6d+lY4XYxcXGoqbVB8+Z1+1IgKysbKSnpMCx3bp06qUNBQR6PK4m3trKzcpCW+hr65a5Th47tIK8gh6dPnvPkfxL3DPoG3BMF6ulr48lj3uvPVlpaWqsHHI0hxuoS1H2uLmGPD2gcMXJpBGVzfamywT5lyhRs2bIFbdu2xdGjRzF37lyYm5vjl19+4RR8tdGqVasf2s5Oz8nhfgLUsmXLGn1RVPf8srLKJoYJDQ3Frl27eH4uXLgAAMjPz+eKq6L4v+/i15AKiorx/nMOWspzF5JljfWTyP1WgL0L7aGlLtgldpYtX4hlKxbi+LEzmDfHTaCxfC8q+iHExMRgasI9YZakpCQMDLohOvqhgCKrmpSUFBSVFKrOKEDCEGNjuMcUYz1pBE/xqWwu01jKZjsLfRwOjsDrzC/I/VqA7eduwVynI9Ra8X7Pde+ihpsPXyAlo2xZpcS3H3Ez9gV0viuTbz96iZz8bxjRq+JlWyvzzyEfLF48Cx06tIOcnCz+WO+Ka1dv8R236nPCD4Y99DDWfgTExcUx1n4Euhvq4sTxswAAE5PusOxnDikpSYiKisK8twl+nfsLrgWHAABaKbfE+Am2UPj/W3oNjU5Yv8ENYXfvc9761iXvg8exdOlcdOzYHnJysvhz0wpcuXITKSnpVe9cC0ePnML8RTOg3qEdZOWaY/UfS3Hj2h3O6iflnTrpDwNDXdiOGQZxcXHYjhkG/e7d4OtTtsyllnZXGPbQg4SEBMTFxTFkWH/YO45CwLlLP32M1SWo+1xdwh4f0Dhi5GgEZXN9qdZMYqNHj8bo0aORnZ2NmJgYXL16FWfPnsXUqVMRFBQEJSUliIiIcM0OWt73BXd5VRXgmZn8ZwRnp8vJcXfV+ZGnetU5P/Zxli5diunTp1f5mez8FcX/8ePHGsdZU19yv0JRlrf74e7zoSguLUVfvS6ctDcfszDD0xc5X8sa6zodKp4MqCG4us3HshUL4XPCD3Nmu3LG8QmLU6fPw811PhYsmI7Qu5Gc9OnTJqB5cxmcOHlOgNEBKiqt8J7PbPq9+5hBW6cr7oZG8tmrYQl7jMJ+jwGKsamjsrnxlM1TB5shJ/8bJv55DIXFxeip3REbpw4DAFyMeIoNJ4Jxb8ciAMDkgabI/VqA2TtO4UveVyjISGGgkSamDDbj+swzdx5ikLEW5H9wFRcP971QVFJAyG1/NJNshps37mL6/yeBc3Ache07N6CtatnM80lJqZg04Vds3LQcu/dsQXJSKiaOn81p3Es0k8C6Da7Q0OgEhmHw9k0G9v79Dzzc9wIAGIbBhEljsHnLKjSTbIaPHz/javAtbNq4/Ydir8qWrbugpKiA8LAgSEo2w7Xrt+H8y/x6OVZ5Ozz2Q0FRHsE3z0CyWTOEhNzFrzOWAgDG2I+A+/Y/0FGtB4Cyyd6mTJqPPza6YsfuTUhJTsMvk+ZxGs4tW7XA5m2r0F5dDUVFRUhOSsPCectx5dKNnz7G6hLUfa4uYY8PaBwxkmo22Nnk5eVhaWkJS0tLlJaW4uzZs4iOjsaAAQOgoKDAdzmX3NxcJCcn/3CA0dG8M0Xn5eXh2bNnkJaWRqdOdbckSGXnx+4eyO5+VxX22L8HDx6gtLSUq+tdaWkp3/OqaweC7uFx0hsYa6qjTQt55H8rROiTJNx/ngq9Tm0w3qrsCznvWyFmePrizccsjLfqgZSMT0jJ4O562FO7I1rKN6/3mAFgxkwnrFi1GKmprxFy8y4cHEdybX///iNu3ghtkFgqEhf3DH/v+Qfz5k7F6VMHcOnSDWhrdcW8eVNx61YYfHwE2wjZ6rEWqq2VEXo7HGlpbyAlKQn97t1gO2YocnPysGbFZoHG1xhiFPZ7TDHWH4ZpXMu/UNks/GWzmKgoXMZa8Z3EdZiZDoaZ/dftWFxMFAttLbHQ1rLSz/x7gX2l26tSWlqKlcv/xMrlf/JsO+UbgFO+3CuyXLt6G9eu3ub7WWF378PSYlSFx/qY+Qkjhk6qcHtdKy0txe9u6/G72/oGOyb7uGtXbsXalbzrmp89HYizpwO50m5cv1Ph5Gx370SgT8/hTTLG6hLUfa4uYY8PaBwxsjW2srkuVdlgDw8Ph5mZGc/TcfZYMvYEMrq6uggNDcW9e/fQq1fZzOOlpaXYunUrvn798QmFwsLCcOfOHa7Jbfbu3Yvs7GzY29tDQkLihz8bqP756evrw9DQENeuXcPZs2cxZswYns+KjY2FpqYmpKWl0aZNG5iamiIyMhKnT5/mmonWx8cHKSl1PzPq94xZ7ZH47iMC7z1BVt5XiIqKQl1FEfNG9YHTAGPOUm1fcr/idWZZt0Kfm/zXoT2w2LHBGuw9jMqe6Kurq2HfAXee7Xduhwu8wQ4ALkvWICUlHdOnT8TQIf2RmfkJu3cfxpo/tgm8R8C5sxfhMG4U7B1HoWWrFmAYBulpb/DvYV/s2nkQr9PfCjS+xhKjMN9jNoqxHjSCsW5UNjfespkQQsgPaARlc30RYaqoLRkbG0NGRgbdu3eHmpoaGIZBVFQUHj9+DH19ffj4+EBcXByhoaGYPn06JCUlMXToUMjKyiIyMhLfvn2DlJQUnj17hufP/5vQwsvLC7t27cK///4LMzMznuP6+flh2bJlsLKy4qz1qqamhtjYWEREREBNTY1nrVf2Oqs3bvDviuPm5oZz587h+vXraNeuXY3ODwBev34NZ2dnpKeno1u3btDT00Pz5s3x9u1bxMXFITU1FaGhoVBWLptEpvxar9bW1tDQ0EBCQgJCQ0NhZmaG0NDQCs+/Ml9vetcof0NTHb5J0CFUKr+IdxkdYaMkzbu8CqmZz195Z7wlP5/iwrpfJ/bbg/O12l+qx8iqM9USlc1UNtcUlc21R2Vz7VHZ3DQ01bK5vlQ56dySJUugq6uLJ0+e4MSJE/Dz80NJSQmWLl2Kf/75h1NgWlhYYOfOnejUqRMCAwNx4cIFdOvWDT4+PpCXl6/iKBUbNGgQtm/fjpSUFBw5cgTPnz/HqFGj4OPjUydL11T3/ABATU0N/v7+mD9/PkpKShAQEIDjx4/j8ePH0NTUxJYtW6Ck9N/60RoaGjh58iSsrKwQERGBo0ePIi8vD0eOHEH37t35hUMIIUTQGsFMtFQ2U9lMCCFNSiMom+tLlW/YBYX9FP/PP/+EnZ2doMMRKvQUv3boKX7TQE/xm4Z6eYof7V+r/aWMRtdRJMKHyuaKUdlcO1Q2Nw1UNjcNVDbXrRpNOkcIIYT89Eqb7sQ2hBBCiFBqwmUzNdgJIYSQ8hp51zlCCCHkp9OEy2ZqsBNCCCHllTbdSgEhhBAilJpw2Sy0DXY7OzsaH0cIIaThNeGn+FWhspkQQohANOGyucpZ4gkhhBBCCCGEENLwhPYNOyGEECIQTbjbHSGEECKUmnDZTA12QgghpLwmXCkghBBChFITLpupwU4IIYSUwzBNd+kYQgghRBg15bKZxrATQgghhBBCCCFCiN6wE0IIIeU14W53hBBCiFBqwmUzNdgJIYSQ8prw0jGEEEKIUGrCZTM12Bsh9dHbBB1Cpaxa6gg6hEo1ExETdAhVupL5WNAhVCm/qEDQIRBSP5rwU3zy4wwd9wk6hErtUTAXdAiVki8V/vGpswseCTqEKmXkfRF0CITUjyZcNlODnRBCCCmvCT/FJ4QQQoRSEy6badI5QgghhBBCCCFECNEbdkIIIaS8Bu52Z21tjdevX/PdZmBggFOnTnGlMQyDkydPwtfXF4mJiZCSkoKRkREWLFgAbW3thgiZEEIIaVjUJZ4QQgghAATS7U5OTg6TJ0/mSW/dujVP2h9//AEfHx+oqalhwoQJyMvLw8WLF+Hg4IDDhw/D2Ni4IUImhBBCGk4T7hJPDXZCCCGkPAE8xZeXl8f8+fOrzHf//n34+PigY8eOOHPmDOTk5AAAEyZMgIODA1asWIGgoCCIiQn/5JqEEEJItTXhN+w0hp0QQghpJE6ePAkA+PXXXzmNdQDQ1tbG8OHDkZycjIiICEGFRwghhJA6Rm/YCSGEkPIE8BS/sLAQ/v7+ePfuHZo3bw5tbW0YGRlBRESEK19kZCQAoHfv3jyfYWFhAT8/P0RERMDcXLiX8CKEEEJqpAm/YacGOyGEEFKeAMbJffjwAa6urlxpGhoa2LZtG3R0dAAA+fn5eP/+PWRkZKCsrMzzGR06dAAApKSk1H/AhBBCSEOiMeyEEEIIAVDrp/jZ2dnIzs7mSZeXl4e8vDxPup2dHUxMTKChoQFpaWkkJyfj4MGDuHDhAqZMmQJ/f3+0adMGOTk5AMDVFb48WVlZAODkI4QQQn4a9IadEEIIIQBq/RT/yJEj2LVrF0/6vHnz+E4sN2/ePK7fdXR04O7ujtLSUgQFBeHgwYNYuXJlrWIihBBCGjV6w04IIYSQujB58mTY2trypPN7u14ZBwcHBAUF4cGDBwD+e7Ne0Rv03NxcrnyEEEIIafyowU4IIYSUV8tudxV1fa8pJSUlAGVj1wFARkYGKioqeP/+PT58+MAzjp09dp09lp0QQgj5aVCXeNKUdNHohN9c50LfQAetW6tAXEIcr9Pf4lrwLezeeRAZGR8aLJYxc+3RRbcLuuhpoLV6a2SkZWBm72k8+SQkJdDPzhom/U3QUbsTFJUV8fn9ZyTEPIfvDh+kv0yvtxhHzxmDzrpd0FmvC1TVW+N9WgbmWszkm3eiqzO0zbqhdYfWkJFrjqyPWUiJT8L5/f54Gh5XbzF+T0OjExzHjYJ1/z7o1EkdklKSSEpKhb9fEP7efRj5+V8bLJbKiIiIYMH86ZgxYxI6dmiHDx8+4cyZQKz5Y5tQxCjs8QEUY70Qkm53Dx8+BAC0a9eOk2ZqaooLFy7g7t27GD16NFf+0NBQAICZmVnDBUkIIYQ0BCEpmwWB1mEXID8/P2hqasLPz69Bj9tWTRWqrZURdOEqNqzzwMplm3Dr5l04/+KAa7f80KpViwaLxdl1MvTM9fEu5S1yvlQ8UZJKO1XM2zIfsopyuOZ7FftX7cWdgFswtOyB7Ze8oNdLr95inOjqDF1zPWSkvENuJTECQFdDFlLik+G/1w8HVu5B8LFLUFVvjT98N6Kvbb96i/F7k5ztMWfeVCQlpWLLZi+sWrEZLxISsXrtb7h64wykpCQbLJbKuP+1Fu5/rUV8fAIWLlqFs2cvYN68qQg4d4RnOSuKjz+KsR6UltbupwZevXqFr195H1okJCTA09MTADBixAhOuqOjIwBgz549XF3j4+PjceHCBXTs2JEa7LUkqLKZEEJIJRqwbBY29Ia9nIiICDg7O1c4MdDP4s6tcNy5Fc6Tfi8sCgeP7MC4iXbYtcO7QWKZaTENGakZAICdV3dDSkaKb77sj1lYNHg+kp4mcaXf8g+BZ9BO/LJiKpYMX1wvMc61mIn3aWUxugfvrDBGAFg7jndiqEuHL8Dr9j7Yzh2L2+dC6iXG7wWcuwSPv/YgO/u/Cv2hgyeQ+CoZS13nwXmyA/bvO9ogsVRER4eFeXOnwu/cRTg4/tdjzC2QgwAAIABJREFUISk5FTu2b4Cj4yicPOlP8VWCYqwnDViwBwUF4fDhwzAxMUHbtm0hJSWFpKQk3LlzB8XFxbC1tcXIkSM5+U1NTTF+/Hj4+Phg1KhRGDRoEPLy8nDx4kUAwMaNGyEmJtZg8TeEplI2E0IIqYQAGt1Pnz7Fzp078eDBA3z79g2dO3fGuHHj4OjoWO0XDjk5OThy5AiCg4Px+vVriIqKQk1NDUOHDsWECRM4K7xUht6wE4601NcAAEXF2o+9rC52Y70qOV9yeBrrAJD2Ig0pCSlQZ9XfmE12Y/1Hfcv/htwvOWiu0LyOIqpaTMxjrsY629mzZZV6bR3NBoulIuMcR0NUVBQ7d3I/HPI+eAJ5efmYON5OQJGVEfb4AIrxZ2BmZoZ+/fohOTkZgYGB+Pfff/Ho0SOYm5tj586d2Lx5M0+lYM2aNVizZg1kZWVx4sQJXL58GaampvD19YWxsbGAzoTUhKioKJauWYCwp8GITgzBzkNboNhCgW9eldbK2H3kL1yPPo9n7+9jxNghDRKjiKgIDFeNx5jHf8Mh4QD6HFgAyRZVVyy7OvfHxDfHoLtwVN0HJSoC7dUTMOjJPgx5eQjG3ovQrEXVkyx2mDwAI975oOui/yaDlFBsDvNzqzEobi8GvzgI6/DtXNurFY6oKFauW4JHL+7geWok9h/ZDqUWihXm79ffAjfCAvDyTTSuh/mjr5U513YxMTEscZuLiEdX8SL9Pu4+uASrARac7Qlp97l+Et/FIDXzUaXH/FGioqLY8udKvH39CJ8/Pscp3/1o2VKpzo9TG8Ieo7DHBzSOGAUlKioKjo6OCA0NhaWlJZycnFBcXIw1a9Zg3bp11fqM7OxsjBkzBl5eXpCSkoK9vT1GjRqFoqIiuLu7Y/z48Xx72X2PGuxNmKRkM7RooYQ2bVXRz7o33HeU/fFdC74l4MiqT0REBC1UlPAl84ugQ+EipyQH+ZYK6KDdEdPWzUS7ru0RczNa0GFBTa01AODD+0wBRwIYGxmgpKQEkfdjudILCgrw8OETGBt3F1BkZYQ9PoBirDcMU7ufGjA1NYWHhweuXLmCqKgoPHnyBGFhYThw4ABsbGz47iMiIoIJEybg/PnzePToEe7fv4+9e/dCR0enLs6eNIAZCyaj/+C+cBgyBZYGwwAAW3fzrwAypaW4eysCv/26Em9f1+4Bck3ozBuBdoN64MqwNThntAAAYL7z10r3aa7WEtqzhuLz09R6ianr/FFobWOM0CGrcNVwLgDAcNecSveRbtcKXWYPQ/Z3MZXkF+CR2yFc7T4Xl7tOwz2HjVCz6w31SdbVjmfe4umwGWKN4QPGw1i3bL+d+zbzzaveoR28/92OXdsPQLtDT+zyPICDR3egXfu2nDybPVbD0socE8bMRNd2JrAb6owXzxM521ntTbh+Lp4PRsiNu/j8qe7rQK6/z8OIkTYwtxiODp3KHgQeObyzzo9TG8Ieo7DHBzSOGDkasGwuLi7GihUrUFhYiN27d2Pbtm1YunQp/Pz8YGxsjBMnTiAqKqrKz/H19UVKSgocHR1x6tQpuLm5YeXKlQgMDISFhQUSEhJw5cqVKj9HoA32iIgIaGpqwsvLC0+fPsW0adPQo0cPGBoaYtq0aUhISODZ5/Pnz9i6dStsbGygp6cHExMTzJw5E7GxsTx5ra2tYW3N/4vXzc0NmpqaSE9P5/zu7OwMANi1axc0NTU5P+XzaGpqIi0tDUeOHMHw4cOhp6eHOXPKCouMjAx4eXnB0dER5ubm0NXVhaWlJZYtW8b5DGEyydkez5PC8Sj+Nk6fOwR5BXnMnv4bwu8JvmFZXYMnDUEL1Za4eea6oEPhkJKRwqHYYzj44F/8dXkHrBwG4Or/2LvP+Ciqto/jv3RIpYUWShAhEHoLSNWAIL0nEJpA8FZJwFtFqoYqDyI9iEjxBoWEDkpR6RBK6EgHhYQeAoRsGiRh93kRs2bZNEiyO8te389nXzAzO/vf0c01Z+acM6t+Y3nQEqPmsrS05IvRAaSkpLB27RajZgEoU7YUDx8+Jjk5WW/dnbv3cXUtjo2NjRGSpVF6PpCMBcaMx8mB1GZD8BnQnSULVnI78g7xcQnMnDyflq2bUrZcab1tox88YvXydZw+9ifq588NlrFKf28ufreV+JvRpMQlcXpqKGW96+DgVjzL9zSePYyzM9aS/CShQDJVGODNX8G/kHjzAalxSVycspqS3nUpXK5Elu+pM/sDLv/fGpKfxOssVyenEn/lNprUDMdUo8HxzbLkVr9BvVk4bxk3I28Tp4pnWtAsvNu0wK18Gb1te/ftyp9nL7Jx7VZSUlLYtG4b5/68RO++aT0RKr/pjt/AXvw3YAJ/X0vrURh1P5rbt+5m+tlFi7rQofO7/PTj2lznfRn+Q/sxc+ZCbty4iUoVx5ix03jvPW8qVHArkM97FUrPqPR8YBoZtQxYm8PDw4mIiKBx48a0atVKu9zW1paRI0cCEBoamuN+0mvM22+/rbPc0tJSu9/Hjx/nuB9FjGE/f/48S5cupWHDhvj6+nLjxg327t3L+fPn2bFjB8WKpU2CduvWLQYOHMjdu3dp0qQJ77zzDrGxsezevZv+/fszf/78LE8CctKmTRsANm3ahJeXF15eXtp1Lz6eZ+rUqZw6dYpWrVrRqlUrHBzSujqfOHGCZcuW0aRJE2rWrImdnR1//fUXmzdvZu/evWzcuJGyZXNfCAra9m27uHbtOg4O9tSq7cl7HbxNqhtMtQbVGPKlP9cvXGfdwoIpWK8i+Wkyk/2+wsrakhJuJWnRrRWFHApjV9iOZ0nPjJZrxjdf0rhJAyYGzeSva/rDCwzNvnBhnj3Tb8QBPH2adpzs7QsTG5tiyFhaSs8HkrHAvAaN7vwgtblgODk74la+DBf+vKxddiviDnGqeDxqVOXu7fsGy5IVG2d7HMqV4PGf/9aK+MgHJKsSKVKjIgl3Hum9583+3qQmPiPyl3CqDGyT75msne2xL+dKbIZMiZEPSFEl4uxZkaTb+j3HKg5ozfPEZ9zdcpSKg97NdL9eP42iRPOaWBW2Jen2QyJX7spVHmdnJ8qVL8u5Mxe1yyIjbqFSxeFZsxp3bt3T2d6zpofOtgDnzl7Es2baELWmLbxQqeLo3O09+r/fG7Vaze4/9jM1aBYJ8Yl6n+/TrzuPHz1m9+/53yvSxcWZihXLcer0Oe2y69cjiY1VUbu2Jzf/GUJpTErPqPR8YBoZdRiwNh87dgyA5s2b661r0KAB9vb22m2yU6VKFQAOHDigUwc1Gg0HDhzAwsIiVxPFKqLBvm/fPmbOnKkzsc7s2bNZvHgx69ev54MP0iYqGj16NPfv32fhwoXaIg7w4MEDevfuzZdffkmzZs2ws3v5GbDbtGmDk5OT9qQgu4ltLl68yKZNm3QetQPQpEkTDh06pD1JSHf48GGGDh3K999/n+sxD4Zw724U9+6mda/bsW03W3/5gz/2rqewfSHmzf7ByOmyV7lWZb7830QeP3jMlPcnkfJMOSf7arWac4fOav+9J3QnE9dMJShkCl90/JTnqYa7Q5Juwpf/5T8fDWL5stXM/naRwT8/M4lJSZR0zHxcf/os9sZ85JfS84FkFAVLanPBcPjn9xCn0r3jq4qNw9HJcHOdZMfGMW1y1WSV7m8zRZWIjWNhve3t3YpT85Ou/N5pYoFlsv4nU0qcbuM1RZWAtZN+psJuxanySXfCOn6Z7X6PDZgJlhYUqVuZ0m3rk/w4+6fBpEv/b/XifDGq2DicMvnv6OjokOm2HtXeBKBYsaI4OztRxeMN3m7SGXv7wixdOY+gqaP54pMgvf31H9Sb1Ss3oC6ARoyTU9pcBbGxKp3lT56ocHbOec4AQ1B6RqXnA9PImJ9UKhUqlUpvubOzs94F4Bs30i4MVqyoP0eWlZUV5cqV4+rVqyQlJVG4sP7fn3S9e/dm69athISEcPnyZerVq0dqaipHjx7l3r17TJkyhRo1auSYXRFj2Bs2bKhzQgBpXxDSrvBD2iNrTp48SYcOHXROCABKlizJ0KFDefjwIUeOHCnwvP7+/nonBADFixfXOyEAaNq0KW+++SaHDh0q8Gx5cfHCFc79eZHBQ/2MHSVbb9SszKRVU0lQJTDBdyyPo/Sv9CuJWq3m4OYDVKjmTvXGOf8o89vYcSP5YkwgP61cxycj9GexN5Z7d6MoUaIYtra2euvcypYmOvoRKSnGuxCj9HwgGQuMRp2312tCanPBSIhP6y7u5Kw7gZuzixPxcQXTlfxlpcQ/BcDWWfdE1MbZnpR4/QtsTb715/zcLSTdjymwTKn/ZLJxsn8hkwOpcfqZ6sz6gGtzN/E0N5nUGp6c+ouUuCRqTR+Sqzzp/61ebNg4uzgRl8l/x/j4hCy2jdeuB5g5bQHxcQk8iHrIwnnLaNfhHb19NWvRmPIV3Qj5aUOusr6s9EwuLrqNmCJFnDOd0NYYlJ5R6fnANDLqyGNtXrFiBa1bt9Z7rVixQu+j4uPTjo2TU+YXLtJrSsbHq2bGzs6OFStW0KtXL06fPs3y5ctZuXIlV69epW3btjRt2jTb96dTxB32zK4slCmTNv4nNjYWgNOnTwPw5MkTFixYoLd9REQEANevX9cbJ5DfateuneW63bt3ExoayoULF3jy5AnPM4w3U9w4zUwUKlSIIkUzn6lWCd6oWZnJq6eSFJ/EhD7jiL4TbexIuWJbKK2x4uiS8wy7+WnsuJGMHT+SVT+vJ+DjMQb97JycOHmWtm3fxqtRXcIO/dutyM7Ojjp1anDwoP6jBw1J6flAMhYY6RIPSG0uKHGqeO7cuodnLQ8un0+bD6BcRTecnB25evGaQbNkJUWVSMLthxSt5U7MhbTJ2hwruGLrbM+TTCaUK9OqFsVquVNnTNoFHVunwhSvW4kyb9dmZ/cp+ZIpVZVI4u1oXGpXQnUhEgD7CiWxcbZHdSlSb3vXt2vjUrsS1cb4AmDtXJgidd/A9e3aHO42KdPPsLCyxOEN/XkEMqNSxXH71l1q1qnOhfNpwxsqVCyHs7MTl85f0dv+4vkrNG3hpbOsZu3qhP3zmN0L59L2oXlhcqwX/w0wYLAPu37fz/17D3KV9WXFxqqIjLxNvbo1OXv2AgCVKlXAxcWZc+cuFchnviylZ1R6PjCNjDryWJsHDRpE9+76T4J48e56foqJiSEwMJAbN24wZ84c3nrrLQCOHDnCtGnT2Lt3L2vWrKFChQrZ7kcRDfbMrl5YW6dFS+/qk35yEBYWRlhYWJb7SkzUH+eT30qUyHxyk//9739Mnz6dIkWK0KxZM8qUKUOhQmlduDZt2sSdO8oYC1KyZAkeZDJLeLMWjanuWYVDYTmPyTCGSjXeYNKqKTxNSGJCn7F5ftxafnNwduBZ0jNSU1J1ltsVtsPbtw3q58/566zhTsZGjwlk7PiRhKzeyMcfjs606BvT2nW/MGZ0ICNG+Os05PyH+uHgYM/q0E1GTKf8fCAZC4zCfivGIrW54Kz9aRPDAgcRfugkTx7H8vmXARzcc0Rv3HM6W7t/eqhYWGBjbY2tnS3PU5/rXHjIb9d+3kON4Z2JOnSJZzHx1B3fh7t7/yQhk7HiG/+ZRT5di8WBRB+7wqXvt+drpps/7eHN4Z15eOgCKY/jqf5lXx7sOUvSLf1M6bPIp2uwZCSPw6/w96K0x5sWqf8m1vZ2PD5xFXVyKsW8PHhjWHsif8r9JLarVqxj+MihHD54jJiYJ4yf+Cl7d4VlOlHc+tBf+ChwMF17dmDblj/o2LUttet4MvLDsQCEHznJxQtX+GzscEb/dxL29oX5aMQQdmzVHVNfwrU47Tq25v2+w/U+Iz8tXbaKUaOGs2//YR49imH61+P5/fe9REYqZxJlpWdUej4wjYxaeazNmXV9z0r6s9GzuoOekPBPT6ks7sCnmzFjBsePH+e7776jdevW2uUdOnTA1taW4cOHs3DhQmbMmJHtfhTRYM+N9AMyatQo/P39c/UeCwuLLLta5tSFIaf9vig1NZXg4GBcXV3ZsmULxYvrzqK6devWV/68/PbN7ImUKu1K2IGj3Lp1l0J2dtSuW4PuPTsQH5dA0PjMH0lSEN7u8Q6ubiUBcCnmjLWtDb0D066GR995wL6NewFwdXNl8qopOLo4svXHX6nWoDrVGlTX2dfR344UyKRuLbu/TYlyrgA4F3PB2taaHoFpdxEe3o7mwKZ9AHg2qckHX39M+G+HuR9xj6T4JEqWL0XLHu9QomwJ1s4J4aGBegQM+2AA47/8Lzdv3mHf3kP4+Op2a33w4BF792R9cm0I589f5rtF/yNg+BDWrV3Cjh17qF6tCgEBQ9i//zAhIcZtyCk9n2QsQHKHPdekNr+aJfNX4FLEmfW/r8DWzoZD+48x6uO0sdader7HpG/H0qDSvzMT/3nr3277X8//iq/nf0XwzB8InllwTx+5GPwrtkUceG/HZKxsrbl34DyHA78DwL17U7y+GcLaKmn/zZPu6c5yrE5OJSUuiacP9ceL5sW1BVuwKeJAyx3TsLSzJnr/OU4HLATArUczas/0Z0flwQA8zSRTalwSyQ/TLjJZ2lpT/Us/HCuXQaPR8PReDDeW/sa1Bbl/ikrwnKW4uDizfc8abG1tObDvMIH/GQ1A994dmTF7IlXLNwLSJqTzH/gJX00ZxawFU7gZcZuhA0ZqG/cajYb3+w5n+qyvOHvlACpVHNt/3cn0yXN1PtO3X3fu3rnH/j0FO5RjxjfBFC3iwtHD27Gzs2XX7gMMfD/rOSSMQekZlZ4PTCOjlgFrc6VKlQCIjNTvvfP8+XNu375NqVKlsh2/DrB3b1o7JrOJ5Zo0aQLAhQsXcsxjMg329K5u6d3vcsPFxYUrV66QmpqqvSsAaQf60iX9rh5WVlba9S8rJiaGuLg43nrrLb0TgqioKEU91m3Thm349OlKb9+uFC9RDI1Gw+1bd1n54xqC5y/jzu3Mr/AXhDa+ban1Vi2dZf1HDQDg3JFz2gZ7qfKlcS6W1lXf79N+me5rWNMhPLid/93DvH3bUOOFjH0/7w/AhSPntA32m5cjObn7ODWa1KJF11bYFrYjPiaOv/68xpLx33Fqj+Eel1e/QVreChXcWLxklt76gweOGr3BDvDpZ0FERt7G378fHdq35uHDxyxc+CNBk2YqokeA0vOBZBTGJbX51ajVar6ZOI9vJs7TW7d1w29s3fCbzrJqJRsZKpqWRq3h9OQQTk8O0VsXsekwEZsOZ/neXb2mFUwotYaLk1ZxcdIqvVV3Nh7izsasG7FHeuh2zX989DIH243PWxy1milffcuUr77VW7dp3TY2rdums2zf7jD27c669t65dY+BPtk/637h3KUsnLv01QK/BLVazRdjpvDFmPwZ0lAQlJ5R6fnANDIag5eXF99//z1hYWHaCVbTnTx5ksTERJ075llJf6RtTEyM9q59uvTHuWU2x8+LTKrBXq9ePXbt2sWGDRvo2bOn3jZnzpzBw8NDe7WjZs2aXLhwgU2bNmknygFYvHhxpl3gXFzSGoT377/8I1WKFy9OoUKFuHDhAomJidjbp02KkpSURFBQEKmpqTnswXC2bNrBlk07jB0DgAm+Y3O13fmj5+haoVMBp8ncxD65m6gt6uZ9vh8dXMBpcuej/3zBR//5wtgxcqRWq5kzdzFz5i42dpRMKT0fSMYCIXfYc01qsxBCCIMwYG1u3Lgx7u7uhIeHs3//fu0z05OTk5k3L+1Cq6+vr3b7x48fExMTQ9GiRbWPPIW0R8AdPHiQ4OBgpk+fjqVl2nzvz58/Z/78+drPyonJNNgBZs2axcCBAxk3bhyrVq2iVq1aODg4cO/ePc6fP8/NmzcJCwvTnhQMGDCAjRs3EhQUxJEjRyhZsiRnz54lIiICLy8vvefnvfHGG5QsWZJt27Zha2tLqVKlsLCwYMCAATmOUbC0tKR///4sXbqULl264O3tzdOnTzl06BDW1tZUr1490zsHQgghFOY1mundEKQ2CyGEKHAGrM3W1tZMnTqVIUOGMHz4cDp06ICrqyv79+/n2rVr9O3bl0aN/u35tGrVKoKDgwkICNB5/Ojnn3/O6dOn2bx5MxcuXNB2gz9y5Ah//fUX5cuXZ9iwYTnmUcRj3XLLzc2NzZs3ExgYyPPnz9myZQurVq3i3LlzeHh4MGPGDIoWLardvkqVKixfvpzatWtrr/67urqydu1a3Nzc9PZvZWVFcHAwdevWZevWrcyfP5958+ZpJ9XJyX//+18+++wzrK2tCQkJYe/evTRr1oyQkJAcTyqEEEIog0atydPL3EhtFkIIUdAMXZsbNWpEaGgozZs3Z+/evfz0009YWloyceJEgoKCcrWPatWqaXuTJSYmEhoaypo1a1Cr1QwdOpT169fr3JHPioVGBhCaHFcXD2NHyFZTlyrGjpAtWwsrY0fI0e8Pzxk7Qo4SU/J/gj8hXlZqcv7P8J34/cg8vd/+Q/1xyeL1Z4xx5i/jS2tlnzs4qwtuxvv88uGzP40dIUdRCU+MHUEIqc35zKTusAshhBBCCCGEEObCpMawCyGEEAVOxrALIYQQymLGtVka7EIIIURGZjgOXQghhFA0M67N0mAXQgghMpLHugkhhBDKYsa1WcawCyGEEEIIIYQQCiR32IUQQoiMzPgqvhBCCKFIZlybpcEuhBBCZCRPOxVCCCGUxYxrszTYhRBCiIzM+Cq+EEIIoUhmXJulwS6EEEJkZMYz0QohhBCKZMa1WSadE0IIIYQQQgghFEjusAshhBAZacy3250QQgihSGZcm6XBboJikuKNHSFb25JOGztCtpLuHjR2hBwVLtvC2BFyZG1pZewI2UpVPzd2BGGqzLjbnXh1fz25a+wI2RqEsvOZQm2OktqcZ1KbxSsz49osDXYhhBAiA40ZT2wjhBBCKJE512ZpsAshhBAZmfFVfCGEEEKRzLg2y6RzQgghhBBCCCGEAskddiGEECIjM57YRgghhFAkM67N0mAXQgghMjLjbndCCCGEIplxbZYGuxBCCJGRGU9sI4QQQiiSGddmGcMuhBBCCCGEEEIokNxhF0IIITIy4253QgghhCKZcW2WBrsQQgiRkRlPbCOEEEIokhnXZmmwCyGEEBmZ8VV8IYQQQpHMuDZLg10IIYTIQGPGE9sIIYQQSmTOtVkmnRNCCCGEEEIIIRRIGuxmyMLCgpEjhnH+3H7iVX9z4+/jzJzxFfb2hY0dTUtJGWs2a5/pq1Gb7nrb/r7nIP3+8ymNWnfDq00PBn70OQcOHzN4ZlDWMczKqFHDWbVqEZcuhfH06U2uXDlk7Eg6TOEYSsYCoNbk7SWEEEKI/GXGtVm6xL+E8PBwBg4cSEBAAIGBgcaO88pmfTuREYH+bNq8nTlzFlO9WhUCAoZQt25N2r7ni0Zj/P+plZaxQZ2a9OraXmeZtZWVzr+X/byWOYt+pHrVygQMGwjA1t/3MPyLiUz/8nM6tfM2WF5Q3jHMzJQpo3n0KIYzZ87j4uJs7Dh6TOEYSsYCYOKF3dy8LrVZCCFENsy4NkuD3cx4elYlYPgQNm7aho/vB9rlNyJuMm/uVHx9uxIautmICZWZsVzZ0nTOpsH98HEMwUt/osob7qxeMhcb67Sfll+vLvgMDuTrOYt4u3ljHB0cDJJXiccwM9WrN+fGjZsAnDy5E0dHeyMn+pcpHEPJWEDMeCZaIYQQQpHMuDZLl3gz08e3G5aWlsyfv1Rn+dJlq0lISKRf3x5GSvYvpWZMSUkhMTEp03Vnzl0kJSWVjm3f0TbWAWysrenw7tuo4uLZc/CooaIq9hi+KL2xrkSmcAwlYwEx4253wvgsLS2ZMX0C9+78ScyjK6xd8wPFixc1diwtY+Z7/vw53wYvpUVHX7za9OCTcVOJeRKb5fZrNm2jYx9/GrXpTq/3h3Ps1J866w8cPobPkECatO3JO1368fXs73j2LLmgv4bBjqGlpSVffz2OW7dOEx19kZCQ77P9nHffbcWpU7uIibnKyZM7adOmhc76RYtmcOrULuLjr7No0Qy99586tYuHDy9pXzExV3n69CZ169bM9+8G8lvJD6aQUcuMa7M02M1MwwZ1eP78OceOn9FZ/uzZM86evUDDhnWNlOxfSsz4x74wGrbuhte7PWjZsQ9fz/6OuPgE7frklBQAChWy03tv+rI/L1w2TFiUeQxNjSkcQ8koxOtn9BcBdO7SjqbNO1GxUkMAVvw438ip/mXMfEt/XsvesKOs/mEuuzf/BMDYKd9muu3vew4SvHQlsyaP5ejv6+ndtQPDR33FvfsPAHgU84RPxk+lR6d2HP5tHSFL53L89Dm+/9/qAv8ehjqGo0Z9TOfObWnZsiuVKzcGYPnyuZluW6lSBdas+YGZMxdSsmQNZs5cyJo1S6hYsZx2m3PnLvPFF5PZunVnpvuoX78NJUpU177mz1/CxYtXOXPmfL5/N5DfSn4whYziNW+wnz59mhEjRtC8eXNq1qxJ8+bN8ff3Z8+ePQDExcWxZMkSBgwYQIsWLahZsybNmjVjxIgRXL6c+8aVt7c33t7exMfHM2XKFFq0aEGdOnXo3bs3R44cASAxMZHp06fTqlUratWqRbdu3Th0yPATbJUpW4qHDx+TnKx/BfnO3fu4uhbHxsbG4LkyUlrGWp4efDykH7OnjmfahM/walCH1Rt+ZeDHn2vvuL9ZqSIAx06e1Xv/8X+u6N+PijZYZqUdQ1NkCsdQMhYMjVqTp1debNmyBQ8PDzw8PNi4cWPm+TQaQkJC6NatG7Vr18bLy4uPPvqIS5cu5emzDUVqc/b8h/Zj5syF3LhxE5UqjjFjp/Hee95UqOBm1FzpjJlv/ZbfGNKvN+XdyuDk6MCnHw8h7OgJ7t6P0tv2j70H6dTWm2pVK2NlZYVv944UK1qEzdsOT0kWAAAgAElEQVTTGptRDx6SnJxCj05tsbS0pHRJV1o29eLKXzcK/HsY6hgOHerHrFmLtJ8zfvx02rV7J9PP6d+/F6dPnyMkZBMpKSmEhm7mzJnz9O/fS7vNd9/9yK5dB4iLi8/xs62srBg40JelS1fl63fKSH4reWcKGdMZszYb22vbYA8JCcHPz499+/bRqFEjhgwZQosWLbh37x7r168H4O+//2bevHnY2Njg7e3NoEGDaNCgAXv27MHX15fz53N/RTA5OZnBgwdz5MgR2rVrR/v27bl06RIffPABFy9eZNCgQRw8eJB3332Xjh078tdff/Gf//yHW7duFdQhyJR94cJZdvd6+vRZ2jZGnrlZaRlDlsxlsF8vWrdsStf2bfh28lhGfDCIa39H8NO6tLG3VStX4q1G9dhz8AizFi7j74ib/B1xk9nfLePg0eNp2Z89M1hmpR1DU2QKx1AyFhAjdbuLiopi6tSp2NtnP5fDpEmTmDhxIiqVCj8/P9q1a0d4eDg+Pj6cOHHilT/fEKQ2Z8/FxZmKFctx6vQ57bLr1yOJjVVRu7anUTJlZMx8qrh47kU9oIbHm9plFcqVxdHBnivX9BvZGo1Gb0JLjUbD5WvXAahW5Q1aNGnI2i07SE19zt37UewLO4p3i7cK9HsY6hi6uDhToUI5Tp3S/5xatfQ/p1at6jrbApw+fZ5ataq/0ud36dIOFxcnVq3a8Ervz4n8VvLOFDLqMOMu8a/lpHNXr15lypQpuLi4EBISQqVKlXTW379/H4DKlStz8OBBihbVHatx5coV+vTpw5w5c1i2bFmuPjM6Opp69eoxe/Zs7d2iZs2a8fnnnzNo0CCaNGnCrFmzsLW11Vm3YsUKJkyYkNevnGuJSUmUdMx84rP0rttZjdM2FFPIOLhfLxb9uIoDh4/zn0F9Afh28liC/m8e/wvZwI+r00483cqUYvynw5k4Yx4OOZyE5ydTOIZKZwrHUDIWELVxJrYZN24czs7OtG3bluXLl2e6zfHjxwkJCcHd3Z3169fj5OQEgJ+fHz4+PowfP57t27dj9cJTLJRAanPOnJwcAYiNVeksf/JEhbOzk8HzvMiY+RL++Tvh+MLfEydHR+ITE/W2b9XUi5nBS+nUzhuPKm+wbvN27kVFU6FcWSBt7G7XDu/y9ZxFzFzwA8+fq+nY9h26d3y3QL+HoY6hk1PacVKp4nSWx8aqcHZ2zDRXZtt6elZ9pc/39+/H+vW/6n3P/CK/lbwzhYw6jFSbleC1vMMeGhrK8+fPCQwM1DshAChdujQATk5OeicEAB4eHjRu3Jjw8HBS/hmbnBujR4/W6drZoUMHbGxsUKlUjB49WntCkHHdlStXXuar5dm9u1GUKFFMJ0s6t7KliY5+9FLfuSCYQkYba2tKlijOkwx/5FycnZj79QT2/rKKFQtnsu7HBexYu5ySJYoBUKlieYPlM4VjqHSmcAwl4+tj9erVHDp0iGnTpmV7hz00NBSAjz76SNtYB6hevTqdOnUiIiKC8PDwAs/7KqQ25yy9q/GLj7ksUsRZrzFlDMbM5/BPT5z4DPPHAMTFx+OYyW+mS/s2DPbryZhJ3/B2Zz8uXfubJg3rUuSf7MdOnmX81FlMHfdfTu39hX2/pk2EOX7a7AL9HoY6hnFxacfpxYaXi4szKpV+l/a4uPgstn35TG+8UZF33mnGkiU/v/R7c0t+K3lnChlFmteywX72bNo44ubNm+e47YkTJwgMDNSOk0sfO7h3715SUlKIiYnJ1Wc6OztTrlw5nWVWVlYUK1Ys23VRUfrjrgrSiZNnsbKywquR7kRPdnZ21KlTg5OZjME2NFPI+OxZMlEPHlK8aBG9dSWKFaVB3ZpUr/omlpaWHDyS1kW15VsNDZbPFI6h0pnCMZSMBcTA3e5u3rzJzJkz8fPzo0mTJtlue+zYMSDtTvCL0mueUhvsUptzFhurIjLyNvUyzKpdqVIFXFycOXfO+HMUGDOfs5MjZUqV5OLVv7TLbt25R3xCIlXf1L8AZGFhwdD+PmwNXcqhHWv56vMA/r5xk0b1agNw4co1qr5ZiZZNvbCysqJEsaL06vIe+w8V7O/HUMcwNlbFzZu3qVdP/3POn9f/nHPnLulsC1C3bo1XyjR0aD/+/PMix1+YbDQ/yW8l70whow4z7hL/WjbY4+LSrgqVKlUq2+1+//13BgwYQFhYGHXr1mXAgAEMHz6cgIAAqlWrBpDpREmZyXinIyNra+ts16WmpuZq//ll7bpfUKvVjBjhr7Pcf6gfDg72rA7dZNA8mVFSxidZdOVasGQlqc+f83bzxtm+//ylq2zY+hsN69Wifp2CeaxJZpR0DE2VKRxDyVhA8nhSoFKpuH37tt5LpdL/e6JWqxk9ejTFihXj888/zzZWYmIiDx48wN7eHldXV731FSumTX4ZGRmZP8chn0ltzp2ly1YxatRw3N3L4+TkyPSvx/P773uJjLxttEwZGTNfr67vsfznddy+e5/4hATmLFpOs8YNcCuj//9UXHwCf0fcRKPR8DjmCZO/DcbR0Z6uHdoAUKdmda79HcGh8JNoNBpinsSy/pff8MwwRr6gGOoYLlu2ms8++0j7OVOnjuWPP/Zl+jmrVm2gfv3a+Ph0wdraGh+fLtSrV4uff16v3cbGxgY7OzusrKywsrLCzs5Ob9JQGxsbBgzoVaCTzaWT30remUJGLTNusL+WY9jTi3BUVJT2BCYzCxYswNbWlo0bN+p1zztzpuCuChrT+fOX+W7R/wgYPoR1a5ewY8ceqlerQkDAEPbvP0xIiPFPnpWUcfH/QvjzwmUa1a9DmVKuJCYlcfDICY6dOkttTw/8enXRbrvgh5VE3r5DLU8PHB0cuHT1LzZv20nJEsWZ/uUog2UGZR3D7Pj59dDORJrebXrMmEAAbt68w+rVmc+SbQimcAwlY8F4caKql7VixQqCg4P1lgcEBBAYGKizbNmyZZw+fZoVK1bkONlceoM3q4amo6OjznZKI7U5d2Z8E0zRIi4cPbwdOztbdu0+wMD3A3N+o4EYM59/fx9UcfH08R9JcnIKbzWqx/99lVZft/6+h0kzF3B8V9rflPiERD6b8DV37kdhY21Ny6ZeLF8wg0J2aXNn1K9dgy8/H863wUu5ez8KO1tbGtarxfjPhhf49zDUMZw58zuKFHEhLOxX7Oxs2b37IIMHjwSgT59uBAdPp0SJtEnlrl+PxNf3A2bM+JLFi7/lxo2b+PoO02m4bdv2My1b/jsp38CBPhw4cIS2bX21y7p1e49ChewM8rddfit5ZwoZ0+W1Npuy17LBXqdOHc6fP09YWFi2JwWRkZFUqVJF74QgMTGRCxcuFHRMo/n0syAiI2/j79+PDu1b8/DhYxYu/JGgSTMV82NQSsZG9Wvzd8RNftmxiycqFVaWllQo58aIDwYxqE8P7Oz+HftY3aMyR0+e4fCxUzx99owypUri16sz/gN8cXbSn+CloCnlGGbn/fd9dYo/wMSJaSdfBw4cMWqDHUzjGErGApDHK/GDBg2ie/fuesudnXXHCV69epX58+fTr18/GjfOvrfO60Bqc+6o1Wq+GDOFL8ZMMXaUTBkzn5WVFaMChjEqYJjeuk7tvOnUzlv77zKlXNn88/fZ7q9rh3fp2qFgJ5nLjKGOoVqtZuzYaYwdO01vXWjoZkJDN+ss27lzPzt37s9yfxkb5llZt+5X1q379eXDvgL5reSdKWTUMvG75HnxWjbY+/btS2hoKAsWLKBZs2a4u7vrrI+KiqJUqVK4ubkRERHBgwcPKFmyJACpqalMnz491+PjTJFarWbO3MXMmbvY2FGypJSM3i3eyvUjXtq0akabVvrjSo1FKccwO7kp/sZkCsdQMiqPs7OzXuM8M6NHj6ZUqVJ89tlnudpv+h3qrO6gx8fH62ynNFKbhRBCiJf3WjbYq1SpwoQJE5gyZQpdunShTZs2lC9fnsePH3PmzBnKly/Pd999x8CBA5k0aRLdu3enXbt2WFhYEB4ezsOHD/Hy8tJO7iOEEMKMGOgq/sWLFwGoV69epuvHjh3L2LFjtV3p7e3tKVmyJA8ePCA6OlpvHHv62PXs7l4bk9RmIYQQr0zusL9+/Pz8qFq1KsuXL+fw4cPEx8dTtGhRPD098fHx0W5jY2PDypUr2bBhAw4ODjRp0oRFixaxcOFCI38DIYQQxqAx0ElBr169Ml1+8eJFLl68SKNGjahYsSKenp7adV5eXmzdupVDhw7RrVs3nfeFhYUBKLp7vdRmIYQQr8JQtVmJLDSKHEAosmNt62bsCCYt6e5BY0fIUeGyLYwdIUfWllbGjpCtVPVzY0cQBpCafCff9xk7qHWe3u+yYnee3r9gwQKCg4OZPn06PXr00Fl37NgxBgwYgLu7O+vXr9d2f7906RI+Pj6ULVuW7du3Y2Wl7N/n60hqc95Ibc4fUpuFEryOtdmYXts77EIIIcQrURs7QNa8vLzo27cvISEhdO3albZt25KQkMC2bdsAmDZtmjTWhRBCvH4UXJsLmjTYhRBCCBMSFBRE1apVCQ0NZfXq1djZ2eHl5cWIESN0us8LIYQQwvRJg10IIYTIwNjj5AIDA/We156RhYUFfn5++Pn5GTCVEEIIYTzGrs3GJA12IYQQIiMzPikQQgghFMmMa7M02IUQQoiMzHicnBBCCKFIZlybLY0dQAghhBBCCCGEEPrkDrsQQgiRgTmPkxNCCCGUyJxrszTYhRBCiIzMuNudEEIIoUhmXJulwS6EEEJkYM5X8YUQQgglMufaLA12IYQQIiMzvoovhBBCKJIZ12ZpsAshhBBCCCGEEBlcvHiR+fPnc+rUKZ4+fcobb7xBnz598PX1xcLCItf7efr0KStWrOC3334jIiICgDJlylC/fn2CgoKwsbHJ9v3SYDdBRQs7GjuCSStctoWxI+Qo/shCY0fIkeNbw40dQYgCoTHjq/ji1UltzhupzflDarN4XRm6Np84cYLBgwej0Who3749JUuWZP/+/QQFBXHlyhWCgoJytZ+oqCiGDBnCX3/9hZeXF3369AHgzp07/PHHH4wdO1Ya7EIIIcRLkQa7EEIIoSwGrM2pqamMHz+e5ORkfvjhB1q1agXAyJEjGTx4MKtXr6Zjx440bNgw2/2o1WpGjhzJzZs3WbJkCS1bttT7HCsrqxzzyHPYhRBCiAw06ry9hBBCCJG/DFmbw8PDiYiIoHHjxtrGOoCtrS0jR44EIDQ0NMf97Ny5k9OnTzN48GC9xjqAtbV1rrrWyx12IYQQIiNpdAshhBDKYsDafOzYMQCaN2+ut65BgwbY29trt8nOtm3bAOjQoQMPHjxg3759xMTEUKpUKVq2bEmxYsVylUca7EIIIYQQQgghXlsqlQqVSqW33NnZGWdnZ51lN27cAKBixYp621tZWVGuXDmuXr1KUlIShQsXzvIzz507B8CZM2eYPn06T58+1a4rVKgQ48aNw9fXN8fs0mAXQgghMpBu7UIIIYSy5LU2r1ixguDgYL3lAQEBBAYG6iyLj48HwMnJKdN9OTg4ABAXF5dtg/3Ro0cATJ48mf79+zNo0CAcHBzYu3cvU6ZMISgoiAoVKvDWW29lm10a7EIIIUQG0mAXQgghlCWvtXnQoEF0795db/mLd9fzk1qdFrpZs2aMGzdOu7x79+4kJiYyefJklixZIg12IYQQ4mVIg10IIYRQlrzW5sy6vmfF0THtMZ1xcXGZrk9ISACyvgOf8TMfPXpE69at9da1bt2ayZMna7vNZ0dmiRdCCCGEEEIIIYBKlSoBEBkZqbfu+fPn3L59m1KlSmXbHT7jfjK7UJC+LOO49qxIg10IIYTISGORt5cQQggh8pcBa7OXlxcAYWFheutOnjxJYmKidpvspHd1v3btmt66v/76CwA3N7cc9yMNdiGEECIDeQ67EEIIoSyGrM2NGzfG3d2d8PBw9u/fr12enJzMvHnzAHRmd3/8+DF///03jx8/1tlPz549KVSoEKtXr+bOnTuZ7qdjx4455pEx7EIIIUQGGrXcJRdCCCGUxJC12dramqlTpzJkyBCGDx9Ohw4dcHV1Zf/+/Vy7do2+ffvSqFEj7farVq0iODhYb8b5MmXKMGHCBCZMmEC3bt1o06YNDg4OHDp0iOvXr1OnTh2GDRuWc54C+ZZCCCGEiZK75EIIIYSyGLo2N2rUiNDQUBYsWMDevXt59uwZ7u7uTJw4kT59+uR6P71798bNzY2lS5eyc+dOnj59Svny5QkMDMTf359ChQrluA/FN9jDw8MZOHBgps/IE6+m8puV+Hz0cGrX8aR06ZJY21hz5/Y9dv2xn4XzlxEVFS35csHCwoIRgf4MG9Yf94rliI5+zPr1vxI0aSaJiUkGzVLHb3Smywvb2XL0xylZvm/tziNM+3EzAPu+/4qizg4Fki8rSjqGppgPJKMwDqnNQgghXnc1atTg+++/z3G7wMDAbGth06ZNadq06SvnUHyDXeS/sm6lKFXale1bd3L3bhSpqal4elZl4Ps+dO/ZkXead+Xhw8c578hM86Wb9e1ERgT6s2nzdubMWUz1alUICBhC3bo1afueLxqNxqB56lerRE9v3QkwrK2sstz+QYyKeaE7sC9kS+LT5IKOlymlHUNTyycZC4ZGJo4TQgghFMWca7M02M3Qwf1HObj/qN7yI4dPsGzFPPr060HwvKVGSJZG6fkAPD2rEjB8CBs3bcPH9wPt8hsRN5k3dyq+vl0JDd1s0EzlShajU/P6ud7+6x83U65UcSqXK8W2sNMFmCxzSjyGppQPJGNBkS7xIr9ZWlry5aTP6ePXnUJ2duzbG8ZnI4N4/Dgm0+29W7dg0rTRVHQvT8SNm3w1/v/Yt+eQdn107BUSE5PQqP/9n7VW9ZbEqeJf64wv812mTxvHwIE+FCpkx85d+/no49E8epT5dykoz9Vq5oXs4JcDJ3mWksJbtary5dAeWfZmW7vrKD9tP0h0jIoKpUswakAnGnlWNmjmdEo5htlRekal5wPTyJjOnGuzzBIvtG7dTJu9sEgR/WcFKoGS8vXx7YalpSXz5+teOFi6bDUJCYn069vDKLlSUlNJfPosx+12Hz/P/pMXmTCkB1aWxvkzoNRjmE7p+UAyFhSN2iJPLyFeNPLTD2jfwZv3WvemtmdLAL774ZtMt63oXo4ff17AvNk/ULl8Q+bN/oH//RxM+Qq6j/7x6T4Ud7f62ldeG8KmkDG3Rn8RQOcu7WjavBMVKzUEYMWP8w3y2Rkt/2Ufe09e5OfJw/kjeBwA4xeFZrrtH0f/5Lt1fzBzRD8OLZtEr9aNCZj5I/ceGqfhpJRjmB2lZ1R6PjCNjOnMuTabVIP94sWLDB06lPr161OvXj2GDh3K1atXdbY5evQo48ePp3379tSrV4+6devStWtXli1bxvPnz/X26e3tjbe3NyqViq+++ormzZtTq1YtunbtyubN+nd9Nm7ciIeHBxs3bmTXrl307t2bOnXq0LhxY7744guioqJ0tu/bty/Vq1fn3r17mX6nGTNm4OHhwY4dO/JwZF6NnZ0txYoVpUzZUrzt3YxZ8yYDsOuP/Tm80zCUnK9hgzo8f/6cY8fP6Cx/9uwZZ89eoGHDugbPtDP8HI3f/5K3hnzF2x9OZvr/thCXyfjg+MSnTP/fFnq1bkytN8sbPGc6JR7DjJSeDySjUAapzTkbMMiHBXOXEhlxmzhVPJO+mknrd1tSrnxZvW19+3bnzzMXWL/2F1JSUtiw7lfOnb2Ib99u+ZLFlDPmlv/QfsycuZAbN26iUsUxZuw03nvPmwoVcn7ecX7asCecwZ1bUa5UcZzsC/Nfvw4cOnuVu9H6jfCd4X/SsXk9qrmXxcrSEp82TSjm7MiWAycNmjmdUo5hdpSeUen5wDQyChNqsJ8/f56+ffsCac+9a9y4MWFhYQwYMEDnmXdLlizh6NGjeHp64ufnR7du3UhOTuabb77hs88+y3TfycnJvP/++xw7dozOnTvTs2dP7t+/z+jRo1m6NPOu13/88QeffPIJFSpUYNCgQXh4eLBlyxb69u2rk6dPnz6o1Wo2bNiQ6edu3ryZ4sWL06ZNm7wcnlfSf2Bvrtw4yp+XDrBu03KcXZz50P9zjh4xTnF4kZLzlSlbiocPH5OcrD/2+87d+7i6FsfGxsZgeWpWLs9HPdvw7cj+TP3QBy/PyoT+cZjBk77Xu+M+N2QHGo2GEX3eM1i+zCjtGL5I6flAMhYUjSZvL3MitTlnzi5OlK/gxtkz57XLIm7cQhUbR42a1fS2r1GzGmfPXNBZ9ufZi9SopbvtshVzuXz9KL/tXkvHzu++9hlzy8XFmYoVy3Hq9DntsuvXI4mNVVG7tqdBMgCoEpK49/AJnpXKaZeVL1Ucx8J2XL2pf6Eo7e/HC39ANHAl8m5BR9WjlGOYHaVnVHo+MI2MGZlzbTaZMez79u1j5syZdOnSRbts9uzZLF68mPXr1/PBB2ljIydOnEi5cuWwsPi364NGo2HcuHFs3LiRQYMGUa9ePZ19R0dHU6lSJUJDQ7G1tQXgww8/pFu3bsydO5d27dpRvrzunci9e/eydOlSWrRooV02a9YsfvjhB2bPns3UqVMBaN++PV9//TXr16/n448/xjJD9+Ndu3bx+PFj/P39jXLCun3bLq5du46Dgz21anvyXgdvihcvavAcWVFyPvvChXn2LPOJ2p7+00C2ty9MbGyKQfKsmhKg8+/OLRtQtUIZFqz9nVW/HWJYN28ATl+JYP2ecKYP74OTfWGDZMuK0o7hi5SeDyRjQTH1rnOGJLU5Z46OaeOVVS90B4+NVeHk5Ki/vZMDKlWc3rYe1d/U/rtHl0EcO3oq7bt0bM2iJd/yfr8A9uw++NpmzK30vLGxKp3lT56ocHZ2KtDPzij9Yrmjve4jm5wcChOf9FRv+5b1qzPr5610bF6fqhXKsH53OPcePaF8qeIGyauTUSHHMDtKz6j0fGAaGTMy59psMnfYGzZsqHNCAGnPtYO0K/zpypcvr3NCAGmPFOrfvz8Ahw4dIjOffPKJ9oQAoHTp0gwcOJCUlBS2bt2qt33Tpk11Tggg7UTC2dmZrVu3kpKSdvJpa2tLjx49uHfvHgcP6hapdevWYWFhgY+PT7bfvaDcuxvFgX1H2LFtN99MX0DAh2P4avIoRn76Qc5vNgAl50tMSsLOzjbTdYUK2aVtY+THVQ3q1AobaysOnr4EpI1vn7x0A41rvkn7psbvhqz0Y6j0fCAZC4o5j5N7WVKbcxYfnwCAs7Nuw9fFxZm4OP0x3fFxCXonyy9ue3D/UZ49S+bZs2Q2b9zB+rW/0NOn82udMbfSM7i46M53U6SIs95FhoJk/8/ft/hE3cZ5XEISjoX1n7vcuUV9BnVqxdiFIbT+eCqXI+/QuOabFHGyN0henYwKOYbZUXpGpecD08iYkTnXZpNpsNeoUUNvWZkyZQCIjY3VLktISGDhwoV0796devXq4eHhgYeHBz16pE1s9ODBA739WFtb613Zh7QTEYBLly7prWvQoIHeMgcHB6pVq0ZSUhI3btzQLvf19cXCwoI1a9Zol926dYsjR47QuHFjKlasmOX3NqSLF65w7s+LDB7qZ+womVJSvnt3oyhRopjOiWQ6t7KliY5+pD0xNBYbaytcizoTE5cIQOgfR4i4G83ADi24ef+h9pWQlHYX4E70Y25HPTJYPqUfQ6XnA8lYUMy5293LktqcM1VsHLdu3qF2nX+PVUX3cji7OHHxwhW97S+cv0ztOrrdUWvVrs6Fc5ez/Ay1Wq13QeR1y5hbsbEqIiNvU69uTe2ySpUq4OLizLlz+v/PFBRnh8KUKVGESxF3tMtuRz0iPukZVSqU1tvewsKCIV3e5pdZozjwQxAThnTn79tRNDTCLPFKOYbZUXpGpecD08iYkTnXZpNpsDs56XfNsLZO69Gv/ueRISkpKQwcOJD58+ejVqvp3LkzH374IQEBAQwcOBAg03GURYsW1ekOl6548bRuSHFx+leZSpQokWnO9OUZ3+Pu7k6TJk3Yv3+/9qRk3bp1aDQafH19s/7SRlCoUCGKFHUxdowsKSXfiZNnsbKywquR7p1qOzs76tSpwcmTZ42U7F/PklN48DiW4i5pd0zuPYxBrdHw8YzldP50pva1+3jaXbB+XwbTa8xcg+VT+jFUej6QjML4pDbnzk8r1hL4yTAqVCyHo5MDX00axZ5dB7VPP8lobehm6tSrSfeeHbG2tqZ7z47UrluDNSFpk+1Vq16FevVrYWNjg7W1Ne07tqa3b1e2bMrbBHmmkDG3li5bxahRw3F3L4+TkyPTvx7P77/vJTLytkE+P11P78b8+Os+bj94THziU+aG7KBp7aq4uRbT2zYuMYnrd6LQaDQ8VsUzdfkmnOwL0aWl/kUoQ1DKMcyO0jMqPR+YRkZhQmPYc2P37t2cP3+enj178vXXX+usO336NCtXrsz0fTExMajVar0Tg0eP0u42ZnZC8vDhw0z3lb78xff06dOHI0eOsGHDBoYNG8bGjRspVqyYUSabK1myBA8e6Odv1qIx1T2rcCjsmMEzZaT0fABr1/3CmNGBjBjhT9ihf/P4D/XDwcGe1aGbDJblSVwCRZz0n+m6cN0fpD5X06p+dQC6tmpIPQ93ve1Cdx7hxMXrTPqgF84Ohut6p6RjmBml5wPJWFBMveuc0khthnmzf8CliDN/7F2Pna0t+/Yd4qNhowDo2bszs+ZOwt2tPpA22dvg/oFMmjaaeQu/JjLiFu/3D9A2nIuXKMb/zfyS8hXcSElJIeLGLUYGjOP3HXte+4y5NeObYIoWceHo4e3Y2dmya/cBBr4faJDPzmhIl7dRJSTSb8ICklNTaVKrCl9/3AeAbWGnmbJsI0d/nAJAQtIzPp+3irvRMcP1r+IAACAASURBVNhYW9GiXjWWTviAQrbGmZRTKccwO0rPqPR8YBoZ05lzbX6tGuw3b94EyLTQnjhxIsv3paamcvr0ab2udOnvqV69ut57Tp7Un6k8ISGBy5cvU7hwYSpVqqSzrk2bNri6urJ+/XreeOMNoqOjGTJkSKbdRAvaN7MnUqq0K2EHjnLr1l0K2dlRu24NuvfsQHxcAkHj/8/gmUwpH8D585f5btH/CBg+hHVrl7Bjxx6qV6tCQMAQ9u8/TEiI4RohSzbv4c9rN2nkWZnSJYqQ9PQZB89c4fjFv6n1Znn6tmsGgEfFsnhU1H88z4HTaV0YW9X3pKizfsO/oCjpGJpiPslYcDQa8z0pKAhSm9N6G0yc8A0TJ+g/13zDul/ZsO5XnWV7dh/McnK2QwfDadGkU77mM5WMuaVWq/lizBS+GDPFaBkArCwt+axfJz7rp38sOjavR8fm/w75KF28CBu/+dSQ8bKllGOYHaVnVHo+MI2M6cy5Nr9WDXY3t7RnBh4/fhxvb2/t8suXL/PDDz9k+965c+eyfPly7Yyw9+/fZ+XKldjY2NCpk/4f2sOHD3Pw4EGdyW2+//57VCoVvXv31ptZ1traml69erFo0SKmTZsGYLTu8Js2bMOnT1d6+3aleIliaDQabt+6y8of1xA8fxl3bmf+XFrJp+vTz4KIjLyNv38/OrRvzcOHj1m48EeCJs3UfzRLAWpY/Q3+vh3FLwdPEhufiKWlBRVLlyDQpx0DOrTAzkhX53NDKcfQVPOBZCwIGrWxE7xepDYLIYTIK3Ouza9Vg/3tt9+mfPnyLF++nGvXruHh4cHt27fZs2cP3t7e/Pbbb5m+z9XVlYSEBLp06cLbb79NUlISO3bs4MmTJ4waNUrvsTEA77zzDh999BHt2rXDzc2NM2fOEB4ejpubG59+mvkVUh8fHxYvXkxUVBReXl64u7vn59fPtS2bdhhsHNmrUHq+dGq1mjlzFzNn7mKj5ninYQ3eaag/8VNuTfnQhykfGudJBUo5hllRej6QjAVBbcZX8QuC1GYhhBB5Zc612WQmncsNBwcHVqxYQYcOHbh06RI///wzERERjBs3jlGjRmX5PltbW3788UcaNWrEL7/8woYNGyhVqhQzZszA398/0/e0bduWuXPnEhkZyYoVK7hy5Qpdu3YlJCSEYsX0JxMBKFu2LG+99RYgV/CFEEKYB6nNQgghxKtT/B32xo0bc+WK/iNF0r24zs3NjTlz5uRq24xcXFyYPHkykydPznW2Nm3avNTENKmpqVy7do0iRYrQtm3bXL9PCCGE4ZjzOLncktoshBDCkMy5Niu+wf46+fXXX3nw4AH+/v5GmWxOCCFEzsx5JlpzJLVZCCGUz5xrszTYDeCHH34gJiaGNWvW4OTkxODBg40dSQghRBYUOA+eKABSm4UQwnSYc22WBrsBzJo1CxsbG6pUqcKYMWMoUaKEsSMJIYQQZk1qsxBCCFNgoVHiM3VEtlxdPIwdwaTFJMUbO0KO4o8sNHaEHDm+NdzYEYQgNflOvu/zYuWOeXq/59/bXmr7b775hvPnzxMREcGTJ0+wt7fHzc2Nzp074+Pjg729vc72Go2G0NBQ1qxZw/Xr1ylUqBANGjRgxIgRmT6bXBiG1Oa8kdqcP6Q2CyV4HWqzkrxWs8QLIYQQeaXWWOTp9bJWrlxJcnIyLVq0YNCgQbRv356nT58yffp0fHx8SEhI0Nl+0qRJTJw4EZVKhZ+fH+3atSM8PBwfHx9OnDiRX4dBCCGEUAxD12YlkS7xQgghRAaGnon25MmT2NnZ6S3/4osv2LJlC2vWrGHIkCEAHD9+nJCQENzd3Vm/fj1OTk4A+Pn54ePjw/jx49m+fTtWVlYG/Q5CCCFEQTLnWeLlDrsQQgiRgUaTt9fLyqyxDtCuXTsAIiMjtctCQ0MB+Oijj7SNdYDq1avTqVMnIiIiCA8Pf/kQQgghhIIZujYriTTYhRBCCAXat28fAB4e/46NPnbsGADNmjXT27558+YA0mAXQgghXiPSJV4IIYTIIK9j3VQqFSqVSm+5s7Mzzs7OWb7v+++/JyUlhdjYWE6dOsWFCxdo2rQpvXr1AiAxMZEHDx5gb2+Pq6ur3vsrVqwI6N6RF0IIIV4Hpj4OPS+kwS6EEEJkkNdxcitWrCA4OFhveUBAAIGBgVm+b/HixSQmJmr/3a1bN7766itsbW0BiIuLA9DpCp+Ro6OjznZCCCHE68Kcx7BLg10IIYTIIK9j3QYNGkT37t31lmd3dx3g9OnTaDQaoqOjOXLkCN9++y29evVi2bJllC1bNm+hhBBCCBNm6uPQ80Ia7EIIIUQGee12l1PX9+xYWFhQsmRJunbtiru7Oz4+PkydOpXvvvtOe2c9qzvo8fFpz7HO6g68EEIIYaqkS7wwKTFJ8caOkC1rS2U/TqhoYUdjR8hRee/Rxo6Qo9ig1saOkC2XSbuNHcHkNXb1yHkjUWDq1KmDs7OzdqI5e3t7SpYsyYMHD4iOjtYbx54+dj19LLswLKnNeSO1OX9IbX79SW02PzJLvBBCCJGBRmORp1d+SUhIID4+XueZ6l5eXgAcOnRIb/uwsDAAGjdunG8ZhBBCCCVQSm02BmmwCyGEEBmoNRZ5er2MGzduaLuyZ5SSksLUqVNRq9W0atVKu9zX1xeARYsW6XSNv3TpElu3bsXd3V0a7EIIIV47hqzNSiNd4oUQQogMDDmvzf79+5kzZw4NGjSgXLlyuLi4EB0dzeHDh4mKiqJSpUqMGjVKu72Xlxd9+/YlJCSErl270rZtWxISEti2bRsA06ZN07kjL4QQQrwOzHjOOWmwCyGEEMbStGlTIiMjOXnyJBcuXCAuLg4HBwcqV67MwIED6devH4ULF9Z5T1BQEFWrViU0NJTVq1djZ2eHl5cXI0aMwNPT00jfRAghhBAFQRrsQgghRAaG7DpXtWpVgoKCXuo9FhYW+Pn54efnV0CphBBCCGUx9W7teSENdiGEECIDU5+cRgghhHjdmHNtlga7EEIIkYHa2AGEEEIIocOca7M02IUQQogMNJjvVXwhhBBCicy5Nstj3YQQQgghhBBCCAWSO+xCCCFEBmpzfnaMEEIIoUDmXJulwS6EEEJkoDbjbndCCCGEEplzbZYGuxBCCJGBOY+TE0IIIZTInGuzjGE3kjFjxuDh4cHt27cN/tkWFhaMHDGM8+f2E6/6mxt/H2fmjK+wty9s8CxZGTVqOKtWLeLSpTCePr3JlSuHjB1JR+U3K7FoybccOradv2+eIPLeGQ4f38HkaWMoVcrV2PEUnw9rWwp//C0O41di225A9pvW98Zh/Eocxq+Ewo4GCpjGFH4rSs9YtERRRv3fJ2w8Hsq+G7+z4VgIIycNx9HZwdjRsqTO40uYLmPWZiGEEFkz59osd9jN0KxvJzIi0J9Nm7czZ85iqlerQkDAEOrWrUnb93zRaIw/SGTKlNE8ehTDmTPncXFxNnYcPWXdSlGqtCvbt+7k7t0oUlNT8fSsysD3fejesyPvNO/Kw4ePJV8WbFv1wMLeKcftLByLYPuOD5pnSVjYGb4Bagq/FSVnLFK8CEu2LqREqeJs+Xkr16/c4A2PSnQf2IW6TWrzYdcRPHv6zGj5hBBCCCGUThrsZsbTsyoBw4ewcdM2fHw/0C6/EXGTeXOn4uvbldDQzUZMmKZ69ebcuHETgJMnd+LoaG/kRLoO7j/Kwf1H9ZYfOXyCZSvm0adfD4LnLTVCsjRKzmdZuiLWXu1I3r0Gu3f9st3W9r2BqJ88QBN9B+tazQyUMI0p/FaUnnHQiH6UKV+aoI+nsmvLHu3ycycuMOm7CfT5T29WzPvZaPmyYs7d7oTxWVr+P3v3HR5F9TVw/JveE0qoCSGIEELoBKJ0AkiTLgm9gyJBfsoLoqAUBcRIB+kl0pEundB7Cz00JRWQJqST/v6xZsmym5C+G/d8nmcfZebOzJnZ2Zy5M/feMWT61G/p188Lc3MzDvkfZ/jnipvYeb2dH38cR9++3TE3N8Pf/wQ+Pt9kuJ1WrZoyY8Z3VKjgxIMHIXz99RT8/U8q5y9aNAMPj7pUrvwea9b8zvDhX6ssHxDgj5OTg/LfRkZGWFiY88EH7bh69WaWY/5u8v/Ro1cXzM3MOHb0FKNHTeSffzTH7NmiMZOnfk1553IEB4Xy/fifOHbkTYu9ZxF3iY2NIzXlzfO36q5NiIqMzlI8hS0+AAwMMGnujUmNRmBsQnLQTeL3roI49XWaNOiAScMOqoubmpN48SAJBxV/u816jMGwVDkMTMxIjY8l+fZFEo5uhuSknMeYRQX1W8kpbcVnaGjI8G+H0tarNWZmplw4fomfv55FxMtItbL2pe35v2mjqOT2PqUdSzF55DQObvNXKbPl3HqKlShGcnKyctqnHX14cCcoX/fjbfqcm6VJvJ7p4d0ZQ0ND5s1TrawtX7GemJhYevfsqqXIVKVV1gubsNCHABQponutAkAH4jMwwLTdIJL/ukHy3UuZFjVyqYtRpTok7FutcrFSUArDb0XXY6zdoBav416rVNYBDu86SnxcPO292mgpsszpc7M7oX1fj/WhQ8fWNGj0MeUruAPgt2penm9nzJjP6dDhI5o06UTFih4ArFw5R2PZChWc2LRpKb6+CylZ0g1f34Vs2rSM8uUdlWVu3LjD2LFT2L37kMZ11KnTEnt7V+Vn3rxlBAbey3JlHWDUV8No286TNi26U6NqEwB+XfqzxrLlnR1ZtXY+c2ctpWI5d+bOWsrqtQsol+6mAYBXl8E4O9RRfnJTGdb1+ABMGnyMceU6xK2eTOz8/wFg1ulTjWUTz/xBrO8w5SduxfekpqaQdPPMmzJHNxG34Ctif/mUuJUTMSztjEnjzrmKMasK6reSU9qKr49PTxq1bsCwj0fQ2d0bgO/mfaOxbGpKCheOX2LSiB958uhphuv8acwvtKrcXvkp6Mo66Hdulgr7v/z9/enXrx8NGzakWrVqNG7cmL59+7JhwwZlmXPnzjF+/Hjatm1L7dq1qVWrFp06dWLFihUqd53Su3r1Kv3796d27drUq1ePzz77jL/++qugdkuNe92aJCcnc+HiVZXp8fHxXLt2C3f3WlqKrHAyMzOlWLGilClbimaeDZk5dwoA/gePazkyBV2Lz7h+GwyLlyXhwG+ZFzQ1x7R1X5KuHCXl0YOCCe4theG3ousxmpqakPA6QW16amoq8a/jcXAui11R3bu5pc8XBbpGX3JzekMG98bXdyFBQaFERkYx7puptGnjqfJ0Oi8MHtyLmTMXKbczfvx0WrdurnE7ffp8wpUrN9iwYTuJiYls3LiDq1dv0qfPJ8oyv/66Cn//E0RFvbtCaWRkRL9+3ixfvi5bMfft78X8OcsJCQ4nKjKayd/70qJVExzLlVUr692zC9ev3mLL5l0kJiay9fc/uHEtEO+e+VeZ1PX4AIxrNyfx7G5SXz2D+DgSDm/CuGJNDGyLv3NZk9rNSPk7RCUvpzwJVX2anpqCYfEy+RG6moL6reSUtuLr1Ptj1i3cyKPQx8RExfDr1KV86OlBKYdSamVfPP2HbX47uXHpFinJup3B9Dk3S4Ud2LBhAyNGjODBgwd4enoyaNAgmjRpQmxsLDt2vGlOumzZMs6dO0fVqlXp1asXnTt3JiEhgZ9//pnRo0errff8+fP07duXS5cu4enpSY8ePYiJiaFHjx5aG9CmTNlSPH/+DwkJ6hfRDx/9TYkSxTExMdFCZIVTn37duRt0juu3T/D79pXY2tny2ZD/49zZy9oODdCt+Azs7DFt0pXEUztIjXieaVlTT28wMFQ0q9OSwvBb0fUYg+4GY1vUlkpuFVWmV3KriO2/FXVNFxBCgH7l5jR2draUL+9IwJUbymkPHoQQERFJjRpV83Q7Tk6OBASob6d6dfXtVK/uqlIW4MqVm1Sv7pqj7Xfs2Bo7OxvWrdua5WVs7Wwo5+TAtXRP5IODwoiMiMKtWhW18m7VqnDt6i2VadevBeJWXbXsCr853Hlwjv2HN9O+Q6ts7knhiQ8AM0sM7exJ+TtYOSn11VNSX8diWMop82WNjDGu0ZikK0fVZpm26Y/lmGVY/W8BhqWcSDy3L3dxZkFB/VZySlvxWdtaUdqxFHdv3FNOexjyiOjIaCpVrZjJkpn7YuJw9t3cweqDS+nU5+O8CFVkg/RhB37//XdMTEzYuXMnxYur3mH85583A3NNmjQJR0dHDAze9KFITU3l22+/Zdu2bcq79QApKSlMmDCBhIQEli9fTuPGjZXLzJgxg5UrV+bzXmlmaWFBfLz6xT3A638Hf7K0tCAiIrEgwyq09u7x5/79B1hZWVK9RlXatPOkePGi2g5LSZfiM207kNRXT0k8vz/TcoaOlTCu05z4HYshPq6AolNXGH4ruh7j5uVbadymIVMWf8+8ib/y4E4QFVycGTV5BIkJiZiYmmBuYaaV2DKjz/3kdIk+5eY0NjaKN2FERKj2NX31KhJb23cP1Jn17Sje0hAZGaUyPSIiEltb9bdx2NhYayxbtWrlHG1/yJDebNnyh9p+ZsbaOi1m1Sf4ERGRyuOmUt7GSmPMLq7vK//dtWN/LpwLAKBt+xYsWvYLA3r7cOTwSbJL1+MDRf9zgNTXsSrTU+Nj4R0DuxpVqQdGxiTdPKs2L2G/Hwn7/TAo4YCx24ekRuV/H/KC+q3klLbis/x3zKfoyBiV6dGRMVja5Gw8qB//9xN3r98jISGROh/WYvKiCQDsXLs7d8Fmkz7nZnnC/i9jY2OMjdXvXxQrVkz5/+XKlVO5IADFK5X69OkDwOnTbwYKCQgIIDQ0lAYNGqhcEACMGDECGxvt/DGJjYvDzMxU4zxzc8WFc2ys9ipJhc3jR084cews+/Yc5ufp8/H5bBzfTxnDqK+GvXvhAqAr8RlVa4DRe27E7/ODFM1NVAEwNMKs3UBSgm6RHKg+aF5BKgy/FV2P8dqFG0z8/EcsrSz5Zc10tl3cyIzVUwk4c4Uz/orvNyY69h1rKXgpBrn7iLyjL7k5TVpz8rffjlKkiK1a5S5321FczL9dcbCzs1WrcKbFpbls9mN6773yNG/ekGXLsjfgZHR0WsyqlV87O1uNzfCjo2I0xpy+7Mnj54iPTyA+PoEd2/axZfMuunl1eHtV/4n4AFITXgNgYK5acTMws3znDXKTOs0VfdcTM36zR+qzh6Q8CcWsy4gcx5hVBfVbySltxRf7b059+9Wp1rZWxEblLN9ePXeduNjXJCclc/HkZTYu3ULrri1zHWt26XNulgo70KFDB+Li4mjfvj3Tpk3D399f5e59mpiYGBYuXEiXLl2oXbs2Li4uuLi40LWrYmCnp0/fDNZw+/ZtAOrWrau2Hmtra1xdc9aMLLceP3qCvX0xTE3VL/Idypbm2bMXJCbK0/WcCrx1lxvXAxk4OPPRz7VFK/EZGWPWshfJf14nNeYVBkVLKj52iidmBmaWGBQtCWaWGLu3xKB4WRLP739TrmhJDMwUTwUMi5TAoEjBvEe+MPxWCkOMR3cfp4u7NwM+GsrnXUbRqU53fMfNoUSZEiQlJhEe9FCr8WmSgkGuPiJv6FNuThMREUlISDi1a1VTTqtQwQk7O1tu3Lidp9sJDQ2ndm317dy8qb6dGzduq5QFqFXLLUcxDR7cm+vXA7n41tgb7xIZEUVY6ENq1HRTTivv7IitnQ2Bt+6qlb918w41aqo2Pa5ew5VbN+5kuI2UlBS1mz//lfgAiI8lJeI5hqWdlZMMipTAwNySlKdhGS5mYF8WI6cqJAUcybCMkqERhsXyv6tTQf1Wckpb8UVHxvB3+BMqV6uknFbWqQzWttb8eTtvxulITUmB3JyHOaTPuVkq7MDAgQOZMWMGZcuWZc2aNYwYMYIGDRowYMAAZXJPTEykX79+zJs3j5SUFDp06MBnn32Gj48P/fr1A1DpRxoVpbh7Zm9vr3GbGU3Pb5cuX8PIyIj69VQHozIzM6NmTTcuX76mlbj+S8zNzSlS1E7bYWSowOMzNsXAyhbjSrWw/PwX5cei73jF7OoNsfz8F4xrNcXQzh4DQ0PMe45RKWtcpR4AFoMmYzF0aoGEXRh+K4UhRlBcZN6/9RfXLtzg1YtXFCtRlMrV3ufKuWs6+R721Fx+RN7Qp9yc3vIV6xgzZgTOzuWwsbFm+rTxHDhwlJCQvO1fv2LFekaPHq7czo8/fsPBg8c0bmfduq3UqVMDL6+OGBsb4+XVkdq1q7N27RZlGRMTE8zMzDAyMsLIyAgzMzO1MTRMTEzo2/eTbA82l2aN32ZG/m8oTuUdsbax4vvJYzjif1L5BpT0Nm/cQc3a1ejSrT3GxsZ06daeGrXc2LRBMf5BFddK1K5THRMTE4yNjWnbvgXdvTuxc3vO+1/renwASVeOYvJhewzs7BUDvHp6k/TX9UzHljGp3Zzk8D/VKvUGxctgVLkOmJgBBhiWKo9po04k/3U9VzFmVUH9VnJKW/HtXLebPiN6UqZcaSytLRk+fhjnjl7g7/AnGsubmplgamaCgYEBxsbGmJqZYGSkqCKWcihFnQa1MDUzwdDQkFof1MB76Ccc2XUsX/dBE33OzdKH/V+dO3emc+fOREZGcuXKFQ4dOsTWrVsZNGgQe/fu5fz589y8eZNu3boxbdo0lWWvXLnCb7+pjnqd1qzu+XPNfwAzmp7fNv++i3Ffj+SLL4Zw6vQF5fQhg3thZWXJ+o3btRJXYVOypD1Pn6p/hw0be+BatRKnT13QsFTB0an4EuN5vXW+2mQDSxvM2g4g6a9rJF09QcrTMFKMTUgOu6dW1qRuC4ycqxL/xzJSX8eozc8PheG3UhhifJuBgQH/+2EkhkaG/DYvZxftQn/oS25Ob8bPCyhaxI5zZ/ZiZmaK/+ET9BswMs+34+v7K0WK2HHq1B+YmZly+PBJBg4cBUCPHp1ZsGA69vaKFgcPHoTg7T2MGTO+Y8mSXwgKCsXbe6hKxWPPnrU0afKh8t/9+nlx4sRZPvrIWzmtc+c2mJubsWFDzv42zZ21FLsithw8ugUzU1OOHTvN8KFjAOjWvQMz50zG2aEOoBjwbWCfkUye+jVzF04jJDiMAX18lJXn4vbF+Mn3O8o5OZCYmEhwUBijfL7lwL4sPEUupPEBJJ7ZjYG5FRaDJoORMclBt4jfuRgAI7cPMWs3kFjfdN3mjE0wrt6IhEOa/l4bYPJhe8w6DAUDQ1JjIkm6e4nEkwWTewrqt5JT2opv7YIN2NjZsHzvIkxNTbh44jJTRir+Pn7UpQVjZnxFq8rtleWPPjig/P/xs8cyfvZYVsz0Y+UsPywszRk5cTiOzg6kpqby5OFTVs1ew9bVO9S2K/KPQWpqamG/6ZBvvv32W7Zu3crChQt58OABM2fOZNGiRXh6eqqUW7ZsGb/88gtdunThp59+AuDSpUv07t2bBg0asGrVKpXy0dHRNGvWjKioKA4fPoyjoyPZYWyau9dBzJn9Az4jBrF9x1727TuCa5VK+PgM4syZi7T8yIvcnhLGhka5Wh6gV6+uytdeDB8+AFNTU+bOXQpAaOhD1q/fluN127xjYJWsWL12AaVKl+DUiXOEhT3C3MyMGrXc6NKtHXGxr+n8cV9uZtKsLb8VRHzBYz1ytbyBnT2WPrNIvHSIhANrMi1r+vFQTGo2JmbW5xCXtXfQ2k0+nKv4IP9/K3khP2P0KOGSq9gsLM1ZtudXTuw/xePQv7GytaJVJ0+q1HRhyU/L+W3++lytH+D0w9xdvGqyrXTuuox0/Tv3+yUy9l/NzfktL3JzfsqL3Cxyn5vzW17kZn2X29xcEP4ruTkwMJB58+YREBDA69evee+99+jRowfe3t456p6SmprK4MGDlWOr3L2r3l1GE3nCjuIdrh4eHmoHPq2vnIWFBQ4OikR88eJFlYuCO3fusHTpUrV11qlTBycnJ86cOcPJkydVBrdZuHChslmeNnw1eiIhIeEMGdKbdm1b8Pz5PyxcuIqJk311ogICMGCAt8qdeoBJkxR3qU+cOJurCnte2L51D149OtHduxPF7YuRmppKeNgjflu1iQXzVvAw/LHE9x9QGH4ruhxjYmISfwY+oFXnFhQvWZz4uNfcvnaXL3uN5cLxS1qNLTMpWuibJ9TpW24WQgiRsYLOzZcuXWLgwIGkpqbStm1bSpYsyfHjx5k4cSJ3795l4sSJ2V7n+vXrOXfuHGZmZsTHZ71LoDxhB9zd3bG0tKRWrVo4OCiafFy6dIkbN25Qo0YNNmzYQHx8PJ06dSIsLIzGjRvj4uJCeHg4R44cwdPTk/3796vcxQfFxcaQIUMAaN26NQ4ODly5coXbt2/j4uLCpUuX5C6+Fshd/Lwhd/H/+/T1Lv7vZXrnavnuj6Wpf16Q3Jy3JDfrB8nN/32Sm3MmO7k5KSmJ9u3bExwczNKlS2natCmgGBNl4MCBXLp0iXXr1uHu7p7ldYaEhNCpUyd69erF/v37efjwYZafsMugc8Do0aOpVq0at27dYv369Wzbto3k5GTGjBnD6tWrMTY2xsrKCj8/P9q1a8ft27dZu3YtwcHBfPvtt4wZM0bjej/44APWrFlD3bp1OXz4MOvXr8fKyopNmzZRrly5At5LIYQQWZGSy4/IG5KbhRBCpCnI3Hz+/HmCg4Px8PBQVtYBTE1NGTVKMdbHxo0bs7y+5ORkvv76a8qUKaNcPjvkCXshJHfxc0fu4ucNuYv/36evd/E35fIuvrc8YddLkptzR3Jz3pDc/N8nuTlnspObZ8+ezeLFixk9ejTDhg1TmZecnIy7uzs2NjacOHEiS+tbunQpc+bMYcOGDdSsWRNPT89sPWGXPuxCCCFEOinShV0IIYTQKQWZm4OCggAoX7682jwjIyMcHR25d+8ecXFxWFhkfrPx7t27zJs37zUMqgAAIABJREFUj4EDB1KzZs0cxSMVdiGEECKdFKTGLoQQQuiS3ObmyMhIIiMj1abb2tpia2urMi06WvFGorRXgb7NysoKgKioqEwr7AkJCYwdOxYnJ6ccNYVPIxV2IYQQIh3pJyaEEELoltzmZj8/PxYsWKA23cfHh5EjR+Zy7ZotXLiQ+/fvs2HDBkxNTXO8HqmwCyGEEEIIIYT4z+rfvz9dunRRm/7203UAa2trgAxf9RkTEwNk/AQe4NatWyxbtixXTeHTSIVdCCGESEf6sAshhBC6Jbe5WVPT94xUqFABULyK7W3JycmEh4dTqlSpTJvD3717l+TkZJYvX87y5cs1lnFxUQwg+K5XiUqFXQghhEhHXs0mhBBC6JaCzM3169dn8eLFnDp1Sm2U+MuXLxMbG0uLFi0yXYezszOffPKJxnl79+4lNjZWOT+tT3xGpMIuhBBCpCN92IUQQgjdUpC52cPDA2dnZ86fP8/x48eV72JPSEhg7ty5AHh7eyvL//PPP7x8+ZKiRYtSrFgxAOrUqUOdOnU0rv/s2bPExsYyderULMVjmJudEUIIIf5rUgxy9xFCCCFE3irI3GxsbMyPP/6IqakpI0aMYOzYsfj6+tK1a1cuXbpEz549qVevnrL8unXraNeuHevWZf1d79mKJ1/WKoQQQgghhBBCFEL16tVj48aNzJ8/n6NHjxIfH4+zszOTJk2iR48eBRqLVNiFEEKIdKQPuxBCCKFbtJGb3dzcWLx48TvLjRw5Mluvhjty5Ei24pAKuxBCCJGOVNiFEEII3aLPuVkq7IWQsaGRtkPIVFJKsrZDyNTLuGiKWlhrO4xMvYyL1nYI72Q3+bC2Q8hU1NpPtR3COzkNzZ++TnnlXvRDTA31L02kSj90kQOSm3NHcnPekNyce5KbdZM+52b9+7aF3tP1CwIhdIU+XhCAft/FF0JbJDcLkTWSm/WPjBIvhBBCCCGEEELoIP28RSOEEEJkQJ/v4gshhBC6SJ9zs1TYhRBCiHRStR2AEEIIIVToc26WCrsQQgiRTooeD2wjhBBC6CJ9zs1SYRdCCCG05OXLl/j7+3Ps2DHu3bvHkydPMDExoXLlynTt2pVu3bphaKg63ExqaiobN25k06ZNPHjwAHNzc+rWrcsXX3yBq6urlvZECCGEEPlBKuxCCCFEOgXZT27//v1MmjSJEiVK4OHhQdmyZXn+/DmHDh1iwoQJnDx5krlz52Jg8ObRwuTJk9mwYQMODg706tWLmJgY9uzZg5eXF6tWrcLd3b0A90AIIYTIf9KHXQghhBBAwV4UODs7s2jRIpo1a6byJP2rr76ie/fuHDhwgIMHD9K6dWsALl68yIYNG3B2dmbLli3Y2NgA0KtXL7y8vBg/fjx79+7FyEi33wkuhBBCZIc+V9jltW5CCCFEOqm5/GTHhx9+iKenp1qz9xIlStCjRw8ALly4oJy+ceNGAIYPH66srAO4urry8ccfExwczPnz57MZhRBCCKHbCjI36xqpsAshhBDppBjk7pNXjI0VjeDSPy1Pq7w3bNhQrXyjRo0ApMIuhBDiP0dXcrM2SIVdCCGE0DFJSUns3LkTgMaNGwMQGxvL06dPsbS0pESJEmrLlC9fHoCQkJCCC1QIIYQQ+Ur6sAshhBDp5LafXGRkJJGRkWrTbW1tsbW1zdI6Zs6cyb1792jSpImywh4VFQWg0hQ+PWtra5VyQgghxH+F9GEXemXMmBGsW7eI27dP8fp1KHfvntZ2SGoMDAwY9cVQbt44TnTkXwT9dRHfGd9jaWmh7dAAqPh+BRYt+4XTF/byV+glQh5f5czFfUyZOo5SpdSffGmDrh9D0N0Y4xKSaD9rJ7W+W8f03RfV5h+8GUK/pQf4YMpGGvywiYHLD3Ly3sMCj7MwnIdvM7cw58yV/Tx8eYsffx6v7XA0ym0/OT8/P1q0aKH28fPzy9L2f/vtN1auXEmFChWYMWNGHu+dEEIIUfjocx92ecKuh3744WtevHjJ1as3sbPL2tOegjbzl0l8MXII23fsZfbsJbhWqYSPzyBq1arGR228SU3V7k+vrEMpSpUuwd7dh3j06AlJSUlUrVqZfgO86NKtPc0bdeL583+0GqOuH0NdjnHRkeu8jHmtcd6qE7eYe+gqVcoUZUSLmgDsuRbEF2uP8WO3BrSvWaHA4iwM5+HbxnzjQ/HixbQdRqZScpna+/fvT5cuXdSmZ+Xp+tq1a5k6dSoVK1bEz8+PYsXeHKu0J+sZPUGPjo5WKSeEEEL8V+Q2NxdmUmHXQ66ujQgKCgXg8uVDWFtbajkiVVWrVsZnxCC2bd+Dl/cw5fSg4FDmzvkRb+9ObNy4Q4sRwsnj5zh5/Jza9LNnLrHCby49endlwdzlWohMoTAcQ12N8fajf1h39g7/+6g2M/cHqMx7ER3Hr0eu835JO9Z82gYTI0UjpR4fuNDz173M2HOJpi6OWJubFEisun4evq1aDVeGDO/L1ImzmDh1rLbDyTfZafqe3urVq5k+fTqVK1dm9erVFC9eXGW+paUlJUuW5OnTpzx79kytH3ta3/W0vuxCCCGEKPykSbweSqus66oe3p0xNDRk3jzVisbyFeuJiYmld8+uWors3cJCFc2iixTRbsuFwnAMdTHG5JQUpuw8T4P3y+BZtZza/Guhz0lMTqFdzQrKyjqAiZEhbWs4ExmXwLE7YQUZska6ch6mZ2hoiO/cyRw7fIq9fxzSdjiZSsnlJyeWLl3K9OnTcXV1xc/PT62ynqZ+/foAnD6t3pXp1KlTAHh4eOQwCpFXDA0NmTbtW8LCrvDsWSAbNiymePGiGZZv1aopAQH+vHx5j8uXD9GyZWOV+YsWzSAgwJ/o6AcsWqTeTSIgwJ/nz28rPy9f3uP161Bq1aqWL/s2Y/oEHj+8zssXd9m8aWmm+5bX2574w1hu/3WWoPAAVq2ZR7FiGW/bs0VjTp7bTejf1zhx9g+aeaq+XeFZxF1CHl8l+GGA8mNja53fu6HVY5hV2ooxOSWFWfsDaD59Cw1+2MToDScybPEG4HcqkI9n7aTBD5voMHsXm87fU84LeR7J/204Qauft9Hgh010nbebbZf+zHWMunYeGhoaMmHKaK7fP8nd0Ass9ZtD0WJFMizfrEUjjpzZyZ+PLnP4zA6aNG+gMt/IyIjR40Zw/voh7odf5HTAPpq3bKScfy/sosrnwd9XCH1+PdNt5gVt5GZdIRV2oXPc69YkOTmZCxevqkyPj4/n2rVbuLvX0lJk6szMTClWrChlypaimWdDZs6dAoD/weNajaswHENdjHHtmTsEPYtg3Mf1NM5PSEoGwNzESG2euamiwdL1sOf5F2AGdPU8TG/o5/14v1IFxo+Zqu1Q3qmg+8ktXLiQmTNn4ubmxurVq1Wawb/N29sbgEWLFqk0jb99+za7d+/G2dlZKuw6YMyYz+nQ4SOaNOlExYqK72Plyjkay1ao4MSmTUvx9V1IyZJu+PouZNOmZZQv76gsc+PGHcaOncLu3ZpvdtWp0xJ7e1flZ968ZQQG3uPq1Zt5vm9fj/WhQ8fWNGj0MeUruAPgt2penm9Hk1FfDaNtO0/atOhOjapNAPh16c8ay5Z3dmTV2vnMnbWUiuXcmTtrKavXLqCck4NKOa8ug3F2qKP8REVG5/t+aPMYZpW2Ylx5IpBjd8JZ82lrDoxRdC2asPWMxrLHboez6Mh1pnVvyJnvvPmx24fMPhDA2T8fAxAZl4D7e6VY91kbTk/w4rtO9Zl1IIDDt3L34ErXzkOfL4fQuq0nH7fsiXs1TwDmLflJY1mn8o4s/20OC+Ysw7X8ByyYvYwVa+biWK6sssxPs76nafMG9Oo2jEqO9ejarh/37z5Qzq9crp7KZ8+ugxw7cpqX/7zKcsw5oc992AtdhT0hIYE1a9YwePBgmjVrRrVq1fDw8GDYsGFcunRJpez58+dxcXFh/vz5BAYGMnjwYOrUqUPt2rUZPHgw9+7d07iNy5cv069fP2rXrk29evX49NNP+fPPPxk3bhwuLi6Eh4cry27btg0XFxe2bdvG8ePH6d27N3Xq1KFevXoEBwdTpUoVBg0apHE7SUlJNGrUCA8PDxISEvLuIBVyZcqW4vnzfzQek4eP/qZEieKYmBRMk+N36dOvO3eDznH99gl+374SWztbPhvyf5w7e1mrcRWGY6hrMT58Gc2iI9f5tFl1HIpqvrNdsaQdABcePFGbd/HfaU8iYvMvyAzo6nmYppyTA/83bgSzfRcTHvZI2+G8U0Hexd++fTvz5s3DyMgId3d31qxZw/z581U+/v7+yvL169enZ8+eBAcH06lTJ3766Se+++47evfuDcDUqVNV3tteUCQ3qxo8uBczZy4iKCiUyMgoxo+fTuvWzXF66yIdoE+fT7hy5QYbNmwnMTGRjRt3cPXqTfr0+URZ5tdfV+Hvf4KoqHdfxBsZGdGvnzfLl6/LUezvMmRwb3x9Fyr3bdw3U2nTxlPjvuW1vv29mD9nOSHB4URFRjP5e19atGqiUtlI492zC9ev3mLL5l0kJiay9fc/uHEtEO+enfM9znfR5jHMKm3FuO3SfQY2ropjMRtszE35X+vanL7/mEev1M/9sH+iqFy6KDXK2QNQ06kElUsX5d7fLwGoXs6eHh4ulLS1xMDAgNrlS9Lw/bJcCn6aqxh17Tzs3b87C+euIDREEc/UiTPxbNkYh3Jl1Mp279mJ69cC2bZ5N4mJiWz/fQ83rt+me89OAFR835le/T7hS58J/HU/CIAnfz/LMHcXLWpHuw6tWLNqc57tT0b0+Ql7oevDHhERwbRp06hTpw6NGzemSJEiPH78mMOHD3Pq1CkWLFiAp6enyjI3b95k+fLluLu74+3tTVBQEEePHuXmzZvs27dP5WnG2bNnGTp0KACtW7fGwcGBa9eu0aNHD6pUqZJhXPv37+fUqVM0bdqUnj178uLFC5ydnfnggw84c+YMYWFhlCun2sT22LFjPHv2jP79+2NqapqHR6lws7SwID5e80XS69fxijKWFkREJBZkWBrt3ePP/fsPsLKypHqNqrRp56kTzdoKwzHUtRh/3HUBx6LW9GnommGZSqWL8kHF0hy7E87sAwF0ql0RgF1XHnD6viKZvU5MLpB409PV8zDNT7MmEhISztKFWRslXdtSDApuW2mVzOTk5AxHke/SpQstW7ZU/nvixIlUrlyZjRs3sn79eszMzKhfvz5ffPEFVatWLZC43ya5+Q07O1ucnBwJCLihnPbgQQgREZFUr16V0FDVN0pUr+6qUhbgypWbVK+e8d+izHTs2Bo7OxvWrduao+UzY2dnS/nyjgRcUd+3GjXU9y0v2drZUM7JgWvpWg0EB4URGRGFW7UqahUKt2pVuHb1lsq069cCcauuer6s8JuDsYkJwUGhzJ+zjD353GVHm8cwq7QVY2RcAo8jYnEt++a3X66YDdZmJtx7/IqyRVRvpreuXp4dAX9xJeQpNcuV4GroM0KeR9KwknrFGRRvgLke/pxhzXLeVUTXzkNbWxscy5XlxtVA5bSQ4DAiI6OoWq0KD8Meq5SvWs1FpSzAjWuBVK3mAkCDxvWJjIyiQ+c29BnQnZSUFA4fPM6PE2cSE63+QMKrdxf+efEPhw/kf4u+gszNuqbQVdjt7Ow4duwYpUqVUpn+5MkTunXrxowZM9QuCo4dO4avry8dO3ZUTps1axZLlixhy5YtDBumGPAqJSWFCRMmkJiYiJ+fHx988IGy/OzZs1m8eHGGcZ04cYKlS5fSpEkTlek9evTg7NmzbNmyhS+//FJl3qZNm4A3TRyFQmxcHCWtrTTOMzc3U5SJjSvIkDL0+NETHj9SPFndt+cwu3cd5ODRLVhYmjN31lKtxVUYjqEuxbjnahDn/nrMysGtVPqma/KzdyMm7zjPb6dv43fqNgBli1jxzcf1mLLzPFZmBd9yQVfPQ4CuXh/TpPmHdG3fn6SkJK3GootGjhzJyJEjs7WMgYEBvXr1olevXvkUVfZJbn7Dxkbxdy0yUnU0/4iISGw19Eu1sbHWWLZq1co52v6QIb3ZsuUPIiIic7R8ZmxsFPG/ve5XryKxtc3ftxNYW6cdV9UnrRERkcq4VMrbWGk8ri6u7yv/3bVjfy6cUwwu2rZ9CxYt+4UBvX04cvhkXoevpM1jmFXaijE2QXGD3sZc9UaZjbkJMfHqN++LWZnT0s2JoasOK98qM6ZtXd4vpd6XOjklhQlbz1DazpKPa72X4xh17Ty0zuDvTWRElPJv0dvxayrrUkURT7FiRbG1taGSy3s0+6ADlpYWLP9tLhN//Jqx/5uotr4+/buz/retpKQU9mfYuq3QNYk3NTVVuyAAKFWqFG3atCE4OJhHj1Tvbrm7u6tcEAB0794dUNzhT3P58mXCw8Np1KiRygUBwKeffoqdnV2GcbVo0ULtggCgZcuWlChRgq1bt6pcrD569IhTp05Rt25dKlasmMke65/Hj55gb19M45MNh7KlefbsBYmJ2n+6rkngrbvcuB7IwMHavZAuDMdQV2JMSErml/2XaVSpLMWtLQh9EUXoiygev4oBIPp1AqEvooiMU7QGsLUwY2bPJviP7crKwa3YOLwtu7/sRAlbxbvjnUtof6A3XTkPTU1NmPjjWI4cOsGzJ89xruCEcwUnZbNBW1trnCs46cxFapoUUnP10UeSm9+IilL87Xj7vLazs1W7yFeUj86grObX92XmvffK07x5Q5YtW5vtZbMirUn+26+ELVIkZ/FmR3R02nFVrRTZ2dlq7CoQHRWj8bimL3vy+Dni4xOIj09gx7Z9bNm8i25eHfIh+je0eQyzSlsxWpoqbnhHvVZtfRf1OlHjzfClx26y73owmz5vx6VJPdn0eTvWnrnD9suqA8slJqcwbvNpnkXFMa9Ps3femM+Mrp2H0Rn8vbG1s1H+LXo7fs1lo1X2z3fqfKKjYnj65DkL566gdbvmautq2NiDcuUd2LAm71vzaKLPubnQVdgB7t69y5gxY/D09KRatWq4uLjg4uLCmjVrAMUd/fTc3NzU1lGmjKJfR0REhHLa7duKp2V16tRRK29paYmLi0uGMdWoUUPjdGNjY7p168azZ884duyYcvqWLVtISUnBy8srw3Xqq0uXr2FkZET9eqqDjpmZmVGzphuXL1/TUmRZY25uTpGiGV9AFoTCcAx1JcbXicm8jInn5L1HdJyzS/kZslLRb3jPtWA6ztmldgFQ3NqCOs4lqVK2GIaGBpy6p6iMNK6suSleQdOF89Dc3Bz7EsVp2boZpwP2KT9b9yiafnfz7sjpgH307NdNq3G+TZ8HtskNyc0KERGRhIaGU7v2m2a3FSo4YWdny82bt9XK37hxW6UsQK1abty4oV72XQYP7s3164FcfGswz7wSERFJSEg4tWup71tO4s2OyIgowkIfUqPmm/OmvLMjtnY2BN66q1b+1s071Kip2kWkeg1Xbt24k+E2UlJSMDDI33a32jyGWaWtGG0tTCljZ8mdRy+V08L/iSI6PpFKpdWfmt9+9AJP13JULGmHgYEB75cqQnNXR47fedNkPz4xma/WH+efmNcs7u+p9vQ+u3TtPIyMjCI87BHVar7pQuNU3hFbWxtu31SPJ/DmXZWyoHjlauC/ZdPiSmuxkObtfwP0HeiF/4Hj/P04d2MCZJU+5+ZCV2G/cuUK3bt3Z9++fVSqVInevXvz+eef4+Pjo3zdzduDxNjYqD+9MTZW9AZI34QjOlpxdymj1+nY29tnGFdm87y9vTE0NGTzZsWADMnJyWzduhU7Ozvatm2b4XL6avPvu0hJSeGLL4aoTB8yuBdWVpas37hdS5G9UbKk5u+7YWMPXKtW4vIl7VaIC8Mx1JUYLUyN8fVurPb5toNipPiGlcrg692YZlUcM1zHrYcv2H75T+o6l6R2+ZIFEjfo/nkYGxvHsP5fqn2+Ga0Yxf6I/0mG9f+SQ/uOajXOt+nzwDY5JblZ1YoV6xk9ejjOzuWwsbHmxx+/4eDBY4SEhKuVXbduK3Xq1MDLqyPGxsZ4eXWkdu3qrF27RVnGxMQEMzMzjIyMMDIywszMTG1QThMTE/r2/STfBptLs3zFOsaMGaHct+nTxnPgwFGN+5bX1vhtZuT/huJU3hFrGyu+nzyGI/4nla+yTG/zxh3UrF2NLt3aY2xsTJdu7alRy41NG3YAUMW1ErXrVMfExARjY2Patm9Bd+9O7Ny+L9/3Q5vHMKu0FWNX90qsOnmLhy+jiX6dyJyDV2nwfhmNg8HWcirB0dthhLxQNN1/8DSCo7fDqfpvH/jY+ERGrDlKYnIKC/s1xzKPuqzp2nm4zu93RowaTDknB6xtrBg/6SuO+p/SOFDclo27qFnLjU7d2mFsbEynbu2oUbMqv2/YCcD5s5cJvHWX0d+MwMLSguL2xRj+xSD27fZXWY99ieK0bt+iQAabS6PPubnQ9WFfvHgx8fHxrFu3Dnd3d5V533//PRcuXMjxuq2tFX8MXrx4oXH+8+cZv64pszthZcuWpUmTJpw4cYLHjx9z584d/v77b/r27YuZmVmO482pXr26Kkf5TGuSPG6cog9laOhD1q/fVuAxpXfz5h1+XbQanxGD+H3zMvbtO4JrlUr4+Azi+PEzbNig/crmz7MmUap0CU6dOEdY2CPMzcyoUcuNLt3aER0Vw8Txml+nUVAKwzHUlRhNjAxpVc1JbfrDl4pKgmMxG5X5C/2vEfoiimqOxbE2N+H2o3/YdeUBJW0smfpJA7X15CddPw+TkpLYs+ug2vS0JvEhQWEa52tbYW86pw2Sm1X5+v5KkSJ2nDr1B2Zmphw+fJKBA0cB0KNHZxYsmI69veIp14MHIXh7D2PGjO9YsuQXgoJC8fYeqlIx2rNnLU2afKj8d79+Xpw4cZaPPnrTz75z5zaYm5vl+9/OGT8voGgRO86d2YuZmSn+h0/Qb0D2xmHIqbmzlmJXxJaDR7dgZmrKsWOnGT50DADdundg5pzJODsoWmIEB4UxsM9IJk/9mrkLpxESHMaAPj7KSlVx+2L85Psd5ZwcSExMJDgojFE+33Jg35F83w9tHsOs0laMg5pUJep1Ar0X7ychKZkPKpZR5tY914L4cdcFzn6nOO/7N6pKdHwin60+wqvYeOwsTGnl5sTAJoqn3/6BYVwKeoK5iRHNf3rTbLt9TWcmdMz56y917TxcMHs5dna27D2yCVNTU04cO8PIT78GoEv39syYNYnK5RQPIUKCwxjS7398/8MYZs7/gdDgcAb3HaWs3KempjKg5wimz/yea3dPEBkZxd4/DjF9iuprKb17d+HRw8ccP3I6x8cxu/Q5NxukamrjoMPatGnDy5cvOX/+vMr0lJQUOnbsyP379/ntt9/w8PDg/Pnz9OvXDx8fH42D+ri4uFC/fn1lc72LFy/Sp08fGjVqxIoVK1TKxsbG0qxZMyIiIjh8+DCOjoqnbdu2beObb75h+vTpdO3aNcO4jx49ymeffYaPjw+BgYEcOXKE3bt3U6lSpWwfA3Nz9cpFdhw8uEkl8af39gVATiSl5H6UbENDQ0Z9MZQhQ3rjXN6R58//4fff/2DiZF9iYnL32qyiFppf2ZUdnbq0xatHJ9yqVaG4fTFSU1MJD3vE8aOnWTBvBQ/DH797JZl4GZf798Dm5zHMK/kZY9TaT3O1/MOX0bSftRNvj8p8k+697EcCw1h9KpDg55G8TkyitJ0Vnq7lGNTEDVuL7DW1cxqauydh+X0emhrmzz1dx3JlOX/9EKuWrWfC2Ny9l/3hy1vvLpRNXzv3zNXyM4I35FEkhYfk5tzn5vyWF7k5P+VFbs5veZGb9V1uc3NByG1uzm/5lZvzkuTmvKX73/hbHBwcCA4O5t69e1Su/GYE1UWLFnH//v1crbtOnTo4ODhw6tQpzp07pzK4zZIlS1T61GVX06ZNcXBwYOPGjbx8+ZLatWvn6IIgL+S2Ql4QUlJSmD1nCbPnLNF2KBrt3L6vQJrN5YauH0PQ7Rgdilpz9YfeatM9q5bDs2o5DUsUvMJwHmoSHvYIh6Lq/Zd1RaG6i60jJDcLIYTIT/qcmwtdhb1fv36cOnWKnj170rZtW6ysrLhy5Qp37tyhefPmHD2a876QRkZGTJkyhc8++4whQ4bQpk0b5bteb968Sb169bh48SKGhtnv+m9oaEj37t2ZM0fRpERe5SaEELqpsPd10wbJzUIIIfKTPufmQjfoXNOmTVm4cCEVKlRgz5497Nixg2LFirFp0yaNI85mV6NGjVi9ejW1a9fG39+fdevWYWZmxoYNG7CyUrzPMK0/XXalNcuztbWVweaEEEJH6fOrY3JKcrMQQoj8pM+5udD1YdeW5ORkWrZsSUJCAqdP52yAhePHjzNs2DD69OnDd999l+NYpJ9c7kg/Of0g/eRyT1/7yX3l3CNXy88K3phHkYh3kdycdZKbc09yc+5Jbs49yc05U5hzc6F7wp7f4uLilK+QSZOamsqiRYt49OgRLVu2zPG6ly9fjoGBAT175m7QBCGEEPlHn9/1qqskNwshhH7T59ys+7doClhYWBheXl40bNgQJycnXr9+zbVr17h16xalSpXCx8cnW+u7e/cux44d4/r161y4cIH27dvz/vvv51P0Qgghckuf+8npKsnNQgih3/Q5N0uF/S0lSpSgXbt2XLx4kTNnzpCYmEjJkiXp1asXw4cPp0SJEtla361bt5g1axY2Nja0b9+eSZMm5U/gQggh8kRqob8X/98juVkIIfSbPudmqbC/pWjRokybNi3P1te1a9dM3wErhBBCt+jzXXxdJblZCCH0mz7nZunDLoQQQgghhBBC6CB5wi6EEEKkU9hf/yKEEEL81+hzbpYKuxBCCJGO/l4SCCGEELpJn3OzVNiFEEKIdPT5Lr4QQgihi/RSFceHAAAgAElEQVQ5N0uFXQghhEhHnwe2EUIIIXSRPudmGXROCCGEEEIIIYTQQfKEXQghhEhHn9/1KoQQQugifc7NUmEXeudlXLS2Q3incjb22g7hncyMTLUdQqZs+izRdgjvFDGusbZDyFS1hbe1HYJW6HOzOyG0RXJz3pDcnHuSm3WTPudmqbALIYQQ6ejzXXwhhBBCF+lzbpY+7EIIIYQQQgghhA6SJ+xCCCFEOvrc7E4IIYTQRfqcm6XCLoQQQqSTkqq/ze6EEEIIXaTPuVkq7EIIIUQ6+ntJIIQQQugmfc7NUmEXQggh0knR68sCIYQQQvfoc26WQeeEEEIIIYQQQggdJE/YhRBCiHT0+dUxQgghhC7S59wsFXYhhBAiHX0eiVYIIYTQRfqcm6XCLoQQQqSjz/3khBBCCF2kjdwcGBjIvHnzCAgI4PXr17z33nv06NEDb29vDAwM3rn87du38ff35/Tp04SHh/Pq1SuKFi1KvXr1GDx4MG5ublmKQyrsQgghRDr63OxOCCGE0EUFnZsvXbrEwIEDSU1NpW3btpQsWZLjx48zceJE7t69y8SJE9+5jokTJ3Lt2jXc3Nxo1aoVlpaW3Llzhz179nDgwAFmz57NRx999M71SIVdCCGEEEIIIYQAkpKSGD9+PAkJCSxdupSmTZsCMGrUKAYOHMj69etp37497u7uma6nQ4cO+Pr6Ur58eZXpu3btYsyYMXz33Xc0a9YMU1PTTNcjFXY9NGbMCGrVqkadOtWpUMGJkJAwXFwaajssFQYGBnwxcghDh/bBubwjz579w5YtfzBxsi+xsXHaDg/Q/RiDXlzTOD0mOpZq5T8s4GgyZlfElk//N5AWbZtSukxJYqJjuX/nL+bNWMLl81e1GptOf8cmpliM8MWwWCkSzx8gYc8q5SzjBu0xdqmLgX0ZDCysSY2LJvX5IxLP7Sf59sUCD7WwnItp9LmfnBBCCKGLCjI3nz9/nuDgYDw8PJSVdQBTU1NGjRpF37592bhx4zsr7H379tU4vWPHjixcuJDg4GDu3btHtWrVMl2PTlfYV69ezaZNm3j48CHx8fFMnz6drl27ajssTpw4wYIFC3jw4AFRUVF06dKFn376KdvrmT9/PgsWLOC3337Dw8MjHyLV7IcfvubFi5dcvXoTOzvbAttudsz8ZRJfjBzC9h17mT17Ca5VKuHjM4hatarxURtvUlO132S1MMR44cxlNvy2VWVaYmKSlqJRV9axNL9tX4yllSVb1u8i+K8QbGytcalaiVJlSmg7PJ3+jk09vTCw0vz7NXJ4n5RXz0i9f4XUmCiwtMbY7QPMe44m4fBmEo9vK+Bodf9cTE8Xfru6THKzEEKIglaQufnChQsANGrUSG1e3bp1sbS0VJbJKWNjY5X/Zlo2V1vKR3/88QfTp0/Hzc2NAQMGYGJigqurq7bDIiwsjBEjRmBnZ8cnn3yClZWVTsSVHa6ujQgKCgXg8uVDWFtbajkiVVWrVsZnxCC2bd+Dl/cw5fSg4FDmzvkRb+9ObNy4Q4sRFo4YAUJDwtnx+x5th5Ghn3+dgpGxMZ2a9eTZ0xfaDkeFLn/HhmWcMf6gLQmH1mHWpp/a/Pjf56pNSzq7F/PPpmPSqAOJJ7ZDAVdKdf1cTE8GncuY5GYhhBDakNvcHBkZSWRkpNp0W1tbbG1VH4AEBQUBqDVlBzAyMsLR0ZF79+4RFxeHhYVFtmO5evUqf/75J6VKlaJSpUrvLG+Y7S0UkOPHjwOwZMkSvvrqK0aOHKkTyffcuXMkJCTwzTffMG7cOEaOHEnLli21HVa2pFXWdVUP784YGhoyb95ylenLV6wnJiaW3j21/ySnMMSYxsTEGEur7P8xyW/uH9TG/YParFjwG8+evsDY2AhzCzNth6Wks9+xgQGmnYaR/Oc1kgOz0bw9JYXUyH/AxAyMtHOvVlfPRZF1kpuzxtDQkGnTviUs7ArPngWyYcNiihcvmmH5Vq2aEhDgz8uX97h8+RAtWzZWmb9o0QwCAvyJjn7AokUz1JYPCPDn+fPbys/Ll/d4/TqUWrUyb2aZ032bMX0Cjx9e5+WLu2zetDTTfdOGgorR0NCQbyZ9yaW7R7kRcoZfV8+kaLEiGZZv4tmAA6e3cTv8PPtPbaVxM9UuQc1aNmLX4Q1cCzrFuVuHmPTTOEzN3vRtnTFvMqevH+B68GnO3/JnxrzJ2NrZ5Hofxkz8gjOBB7n84BjzVs6gSDE7jWVLli7BQr9fOHx5F3eeXqTDJ21zte3c0tq5aGCAyUe9sfx6KZbjV2Hm/SVYav4eTJp0xnL8apWP1ZSNmLbrryhgZYtp18+x+Go+luNXYzFqDiaNO2UrnP/CeVgQ/Pz8aNGihdrHz89PrWx0dDQANjaa98vKygqAqKiobMfx6tUrvv76awDGjRuHkZHRO5fR2Qr706dPAShRQvvNYtN78uQJAPb29lqO5L/LvW5NkpOTuXBRtf9yfHw8167dwt29lpYie6MwxAjQtkMrAsPPcyv0HBfvHGXST+OwsbHWdlgANGmpGDfh0cO/WbRmFldDT3E15BT7z27R+kUA6O53bNygPYb2ZVX6rGfIwgosbTCwL4tJs64YvV+TlOBASErM/0Dfosvn4ttScvn5L5PcnDVjxnxOhw4f0aRJJypWVDSrX7lyjsayFSo4sWnTUnx9F1KypBu+vgvZtGkZ5cs7KsvcuHGHsWOnsHv3IY3rqFOnJfb2rsrPvHnLCAy8x9WrN/N8374e60OHjq1p0OhjyldQ9N/0WzUvz7eTGwUV4/D/DaJV2+Z0+agPDaopRnqetWiqxrLlyjuw2G8Wi+asoEaFhiyas4LFv83GoVxZAIrbF2Ox3yw2r9tOrfca07lVbzwauvPF/71p4bXi1zW0/KAzNZwb0vLDLlhYmDPl529ztQ9Dv+hPizZN8Go7kKY12wPw88IpGsumpqRw+vh5/m/4BB4/fJKr7eYFbZ2LJo07YVzFnbilE4j95XMAzLqN0Fg28cQOYqcOUH7iFn9DakoKSddOAWBgak7qs3Ber5xC7NQBvN7wC8buLTFu0C7L8fwXzsOsyG1u7t+/P4cPH1b79O/fP99jTxMbG8vnn39OcHAwgwYNol27rH3POtckPq3vWBoXFxcAHBwcOHLkCAABAQGsWLGCK1euEBkZScmSJWnZsqWyOVx6W7du5ciRI9y+fZtnz55hZmaGq6sr/fv3V7v7Hh4eTosWLejSpQvDhg1jzpw5XLhwgVevXjFu3DimT5+uLNuv35tmqIcPHwZQLqupz5ynpyeAch9ExsqULcXz5/+QkJCgNu/ho79p0KAeJiYmJCYWfKUjTWGI8erlG+zdeZDgoDBsbKxo1rIx/Yf2xKNBXbq17UdsjHYHTavwvhMAP8wcT0hQGONGTsbUxIQBw3vj++sUTIyN2bbxD63Fp4vfsUGREpg2/4TEY9tIffUMgyKZV5osv5it7OeempxEcuAF4nevKIhQVej6ufg2ea2bOsnN2TN4cC+mTZurbNE2fvx0AgNP4uTkQGjoQ5Wyffp8wpUrN9iwYTsAGzfuYOjQPvTp8wlTpyoq+b/+qrhB5+397idvRkZG9Ovnja/vwrzcJaUhg3vz49TZyn0b981U7t05o3HftKWgYuzZrxvzfJcSFqJY50+TZnP88h4cHMvwMPyxStluPTpy89ptZdegnVv20ntAd7r16MA83yWULlsSM3MzNq/dTmpqKn8/esqRgyeo4uaiXMe9O3+qrDMlJYX33nfO1T549e3CrzOXE/7vPvhOmcehCzso61iaR+F/q5R99vQF61f+rth2cnKutpsXtHUuGru3UOThl4obmAkH12P55VwS7OxJjXie6bIm7i1I+TuYlId/AZD68imJJ3cp56c+DSfp5hmMnKuSdGZvluL5L5yHWZHb3Kyp6XtGrK0VDxQyeoIeExMDZPwEXpPY2Fg+/fRTLl++zMCBA5VP2bNC5yrs9evXx8fHh+3bt/Pw4UN8fHyANwdk8+bNTJw4EUtLS5o3b07JkiW5d+8efn5+nDx5ks2bN6scvMmTJ1O5cmU8PDywt7fn+fPnHDlyhBEjRvD999/Tu3dvtRhCQkLw8vKiYsWKdOrUiejoaJo2bUpUVBQXLlzgwoULdOnSBQcHB0BxAmjqEyFyxtLCgvh49UoSwOvX8YoylhZERGivMlwYYuzyUR+Vf2/btJs7gfcYM+ELBn7am4WzlmewZMGwslY0J4qJiaV/l8+UA5D57zvGoYs7+N/4z9m+abfWBgDTxe/YtOMQRXI/k7W+4K83zsLA2BQD26IYuX0AJqYYmFmQGpv9Jly5oevn4tukD7s6yc1ZZ2dni5OTIwEBN5TTHjwIISIikurVq6pVJKpXd1UpC3Dlyk2qV89ZV4OOHVtjZ2fDunVb3104m+zsbClf3pGAK+r7VqOG+r5pQ0HFaGNrg0O5sty4FqicFhocTmRkFK7VKqtVlFyrVVYpC3Dz+m1cqykqQoE37nL00El6DejOmhWbKF2mJC3bNGPlojUqy3w2ahAjvhqCtbUVcbFxfDl8fC72wRqHcmW4df2OclpY8EOiIqNxcausVmHXJVo7F80tMSxSgpRHD5STUl8+IfV1LIaly5OcWYXdyBjjWk1I8N+UcRkDA4ycq5L8142My6TzXzgPs6ogc3OFChUARd55W3JyMuHh4ZQqVSrL/dejo6P59NNPuXTpEkOGDGHMmDHZikfnKuweHh54eHhw4cIFHj58yMiRI5Xz/vrrL6ZMmUKFChVYs2YNxYsXV87buXMnY8eOZe7cuUyYMEE5fc+ePZQrV05lG9HR0fTs2ZM5c+bQrVs3zM3NVeYHBATw2Wef8eWXX6pMHzlyJPPnz1deFKQfPVYq7HknNi6Okv9W5t5mbq7o46ztV2oVhhg1WTrfjy/GfEbzVo21XkmKj1NUevdsO6AyWnhkRBRHD5ygs/fHVHi/PA/uB2slPl37jo1qNMLoveq8XjkZUrL2ZCMl5M1FWNKV45h9MhLzIZOJm/9/8Domv0LNEl06F98mo8Srk9ycdTY2Vv9uW/XGWEREJLa26t1AbGysNZatWrVyjrY/ZEhvtmz5g4iIvN/3tG4sb6/71atIbG11ow9rQcWYNmBvVGS0yvSoiCisNXT3sbK2UisbGRFFpSoVAcXfna0bdjHpp3GM/2E0xsbG7Ph9D7+v36myzOK5K1k8dyWOTg549e5MyIOcj0uUduNcU1zWNprzn67Q1rloYKqooKW+jlWZnvo6Bswyr7wZuXmAkTFJN05lWMa0TV8MLKxIPJ21Fob/hfMwqwoyN9evX5/Fixdz6tQphg0bpjLv8uXLxMbG0qJFiyytKyoqiiFDhnD16lWNOSwrdLYPuyYbN24kMTGR8ePHq1wQAHTq1ImqVauyZ4/qk6e3LwhA0cyha9euREZGcuOG+h0se3t7RozQ3BdF5L/Hj55gb18MU1NTtXkOZUvz7NkLrTY1h8IRoyZJSUk8/fsZxYppf4Cgvx8r+r891zA6/NMniml2RbT32kGd+o6NjDFr05fk+1dJjX6FQbFSik8RRX9dAzNLDIqVAvPM3/iQdPUEhjZFMa5avyCizjwWHToXtW3nzp1MnDgRLy8vatasiYuLC/Pnz8+wfGJiIkuWLKFt27ZUr16dBg0aMHr0aMLCwgow6jckN6uKilLcDHu70mBnZ0vkWxfKivLRGZTNfkuY994rT/PmDVm2bG22l82KqChF/G+/ErZIkZzFmx8KKsboaEWFzeatmzA2djZER6l/zzHRMWplbe1siP5/9u47rKnr/wP4OwEEmSICKjiwKoiKoICoRRRwUbVCZVnF1brAra1+a+uo1dpl3da24qiCUgStixZEFFSm4gDcggxRBBmCjCS/P/IjEhMQDMm9mM+rj8/T3HtI3sSYzz33nnvO/39eHD60w087vsXy+d/AvIMd7CyGQ1tbCz/t+Fbq62dn5SAqIgZ7j+wAh8N5p9/hZdlLqb9D3VxsxdRnUVAlPFHPeaPecjS0gMqGT+Kr2bqi5nocUFUpdX+r0VOg0sMar/atf+tz1XofPodsNHDgQHTt2hXx8fGiyVYBoKqqClu2CFfi8fb2Fm0vLCzE/fv3UVhYKPY8xcXFmDZtGq5du4b58+e/U2cdYOEV9oZcuyac/Ony5ctISUmR2F9VVYXCwkIUFRVBX194EJidnY09e/bgypUrePLkCSorxf+R1E5UU5eFhYXUg3SiGEnJqRg5chjs7awRG/d6jUN1dXX069cbFy9eYTCdUEvIKE0r9VZo39EI15IaN9RKnm6kpMF3GmDc0VhiX/uORgCA5wWFEvsUhVV/x2qtwNHWg6p5f6ia95fYrWrtCFVrR1RG/IWauJMNPg8AcFozf+WETZ/FNyl64rgtW7YgJycHenp6MDIyQlZW/Vcq+Hw+5s+fj+joaPTo0QN+fn54+vQpzpw5g9jYWAQFBaFbt24KTE+1+U3FxSXIysqGjU0fXL8uHHpqZtYZenq6uHkzXaL9jRvpcHISn6XZ2ro3oqPjmvzaM2d+iuvX05D4xmSZzaW4uASZmdmwse6D1NRbAF7/bjduSP5uTFBUxtKSUuQ8zkUfq15Iv3kbgHBCL11dHaTfuivRPv3mHQz60E5sW+++Foi7EA8A6NuvFzLS7uJ8pPDqa8GzQgQfPIZfdq2vN4OKigo6dDSGplZrvCwrr7dd/b9DGXIe58Gyrzkybt4BAJh2MYGOrjbupEn+DmzC2GfxVTn4L56B28EM/CfC4dIcfSNwNDTBz6//u5tjaAKVrr2kTxjL4aDVuM+g0qknXu1dC0FZcaPjvA+fw8ZSZG1WVVXF+vXrMWPGDPj7+8PNzQ2GhoaIiYnB3bt34evrCzu71+/joUOHsH37dgQEBIiNQAsICMDNmzfRuXNnCAQCqSfj3d3dYWpqKrFdLE/z/WryV1ws/AD//vvvDbYrLy+Hvr4+srKy4OnpiZKSEtjb22Po0KHQ0dEBl8tFeno6oqKipE4oxZZZZpXV0ZATWPHlfCxY8JlYR+mzmZOgpaWJw8FhDKYTYnvGNvp6eFEk+YW/dKU/1NTUEBURI+WnFCvyzHn8r3QJxk8cjd2b/xRNPGZoZACXMU54eC8TWQ+zGcvHqr/jqkq8Ct4ssZmjpQv1cTNRc/caapKjhQcLauoAB5Jn8DkcqNkLZ4/lZd+TeC55aQmfxTcpetK59evXo3PnzjA1NcWxY8ewcuXKetueOHEC0dHRsLOzw969e0Ud2LFjx2LWrFlYu3at1CVq5Ilqs6Q//zyMpUvnIibmMp4/L8L69Svx77/nkZkp+Z126FAoFi+eDS+v8Th27DQ8PNxgY9MXM2e+vhKjpqYGLpcLFRUVCAQCqKurg8/ni43yUVNTw5QpE7Fu3c9y/d3++PMQli/3x/mYS3j+vAgbN3yFiIhoqb8bUxSVMehAKOYsnI7LsYl4UfQCK1YvQkxUHHIe50q0PXbkH8wKmIpxHqNx5kQkxox3RZ9+llgyT3irSEridSxeMQ+Owwbh4vnL0G/bBj5TPHAjVdj5NGjXFk4ug/HfmRiUlpTC7IMuWLFmMRIvp8jUSTp6MAyfz5+K+LhkvCgsxrKvA3Dx3GXkPM6T2l60vBeHAzVVVbRSbwVeDQ88BiahY+qzWJMUBTXH8eA9vAVBRRlajZiEmrvXIHjxrN6fUbN1Be/xHclOPZcLdQ9/cAw7oiJwHfAOc8y8D5/DxlB0bbazs0NwcDC2bduG6OhoVFZWomvXrlizZg18fHwa9Rw5OcK5FLKyssQmbq3L3t7+/eqw187Yd/nyZbRt2/at7fft24cXL15g06ZNmDBhgti+3377TTSD7JveZUgHlyu8u6Cmpkbq/tLS0ibNJChPkyZ5oHNn4aQ8tUN+V6wQng3KysrB4cPHmIyHmzczsHPXPgT4z0DI0d9x5sw59LLogYCAGYiJuSSaSZcy1i9g6eewsbXC5dhE5GY/gZZWawwb4YjBjva4mnQd+34PYjQfILxn6oc1W7Hu5/8h+HQgjgWdgJqaGnymfQI1NTWs/9+PjOZj1d8xnwdeWrzE5tpZ4gWF+aL93PZdoDFjNWpuxUPwPBeC8jJwdNtCte9gcA1NUH01RuzednlrCZ/FNyl60rnBgwc3um1wcDAAYOHChWJXm52cnGBvb48rV64gMzMTXbp0afac9aHaLOnHH3eiTRs9xMb+A3X1VoiKuojp0xcCAHx8JmD79o1o1044qdyDB5nw9p6FTZu+xm+//YSHD7Pg7f25WKfj1Km/MHTo66vwfn5euHDhMkaOfD0kc8KE0dDQUJf7d9OmH7ZDv40erlw6DXX1VoiMugC/afPf/oMKpKiMu37dCz09XRyPPIRW6q0Qe/4KFs8RLm/18UQ3fPfz1+jTRfj3lvUoG3OmLsFX3y7DD1vXIiszB3P8Fos6VckJ17Bq+Xf437dLYdKpAypfVSHhUhK++WIDAOH9u5/4foyvv/sCrVq1QlFhEc5HxuHXTTtl+h1+37ofem108XfEfrRSV0NcTAKWz/saADD2k9FY+9NKDDBzErW//vj1yI8NW7/Bhq3fYPuPe7D9x4ZP2MkDU5/F6ovHwWmtjdazvwNU1cC7fwOVocJVGVSshkB93Oco/27a6x9QVYOqtSOqzhyQeC5uZ3OoWg2BoLoKmotfX33lZWWg8qDkyhbSvA+fw8ZgYkLY3r17Y/fu3W9tN3/+fLEr67WaawWSFtVh79evH27duoWUlBSJZV+kqZ3ZT1rbpKSkZs1Wu0yAtGF8WVlZKCkpYU2Hfdo0b7HCDwBr1ghnK7xw4TLjHXYAWLJ0NTIzs/HZZ5/CbYwLCgoKsWNHIFav/ZE1E0KxOWN8XBJ6mH+AT3zGQ19fDzweH48eZOLH9Vvxx86DqKpn9nNFO3owDEWFLzDTfwoWfDkHAgEf15JuYNncVbiacJ3peKz+O64Pv6QQNakXwe1iAW4vO0BdQziE78kjvIo5Bt71pg+zlUVL+SzWxda/28rKSqSmpkJTUxP9+0veGuHo6CiaLV2RHXaqzZL4fD5WrvwOK1dKroUcHByO4OBwsW3//ReD//6rf7RJ3Y55fUJC/kFIiPyXwuTz+fhixbf4YoX0e1rZQFEZ+Xw+Nqz+BRtW/yKx7/jfp3H8b/FluS6cu4QL5zzqfb5jwf/Uu5xp4fMifDrhc9kCS8Hn8/HDmi34Yc0WiX0nQ8/iZOhZsW0WRnYS7ZjC2GdRIEBVxF+oipCcK4J3PQ7lb9bZmmqUb/xM6lPxH6Xj5TeNu1pbn/fhc9gYbK3NitCiOuyffvopjh49ig0bNsDc3Fxi0pqKigrcvn0b1tbWACBa2iUpKQnDhg0TtQsPD8eFCxeaNZu2tjbMzMyQkpKCBw8eiO4hrKysxHffSRZsJjWm8DONz+dj86+/YfOvvzEdpV5szvjfmfP478x5pmM0yn+novHfqWimY0jF5r9jABC8eCZZ6MtLpd8jx5CW9FlsLiUlJVJnJ2/KGrDSZGVlgc/nw9TUFCoqKhL7azvpjx49eufXeBdUmwkhhBD5aVEd9u7du2Pt2rVYvXo1xowZAycnJ3Tu3BmvXr1Cbm4uEhIS0L9/f/z5558AgEmTJuHYsWNYsGABRo8eDQMDA6SlpSEhIQGjRo1CREREs+abMWMGvv76a/j6+mL06NHgcrmIi4tD27ZtYWRk1KyvRQghRD5kHXa3f/9+qfeqvTkZTVOVlgrvbawdgv6m2u217RSFajMhhBB5Y2JIPFu0qA47AEycOBGWlpbYu3cvEhMTERMTAy0tLRgbG2PixIkYP368qK2FhQX27duHX3/9FefOnYNAIEDv3r0RGBiI3NzcZj8o8PLyAo/Hw/79+xEaGgp9fX24ublh4cKFGDt2bLO+FiGEEPmQdWKbqVOnwt3dXWK7LFfX2Y5qMyGEEHlS9KRzbMIRKPMNAS2UhkZnpiM0qIav+JlC3zeddNg/G7K6CruXV7r3QnJ2VLYpXuHIdIQG9dnBjmWaGvLweWqzP+dQExeZfv5CjvRJ0xqjdpZ4aVfj7969i7Fjx6Jnz5745x/J+wsjIiKwYMECfPbZZ1i+fPk7ZyDvhmrz+49qs+yoNsuOavO7kaU2M43LdABCCCGEvF3nzp3B5XKRnZ0tdQml2sncunbtquBkhBBCCJEX6rATQgghdQhk/CMv6urq6NevH8rLy5GSkiKx/+LFiwCEa7oSQggh7xO21mZFoA47IYQQUgcfApn+yJO3t3CVjy1btqCq6vWSeDExMUhISICDg4NCl3QjhBBCFIHNtVneWtykc4QQQog8Kbqwh4SEIDk5GcDrYe2RkZHIyckBAAwYMACenp4AgI8//hhnz57F+fPn4eHhAScnJzx79gynT5+Gnp4evvnmG4VmJ4QQQhShpXe6ZUEddkIIIaQORc/FmpycjLCwMLFtGRkZyMjIED2u7bBzuVxs27YNe/fuRVhYGPbv3w9tbW2MGjUKixYtklgDnRBCCHkfKPM86dRhJ4QQQhj0/fff4/vvv290+1atWmHOnDmYM2eOHFMRQgghhA2ow04IIYTUoczD7gghhBA2UubaTB12QgghpA6BEh8UEEIIIWykzLWZOuyEEEJIHcp8nxwhhBDCRspcm6nDTgghhNShzMPuCCGEEDZS5tpM67ATQgghhBBCCCEsRFfYW6AaPo/pCETOHpcWMB2hxXMxtmI6wlvpfX+R6QgNKv1rNtMRGKHMw+7Iu6Pa/P6j2iw7qs2yo9qsfKjDTgghhNShzMPuCCGEEDZS5tpMHXZCCCGkDmWeiZYQQghhI2WuzXQPOyGEEEIIIYQQwkJ0hZ0QQgipg6/E98kRQgghbKTMtZk67IQQQkgdyjzsjhBCCGEjZa7N1GEnhBBC6lDms/iEEEIIGylzbaYOO5mtC9sAACAASURBVCGEEFKHMp/FJ4QQQthImWszTTpHCCGEEEIIIYSwEF1hJ4QQQupQ5mF3hBBCCBspc22mDjshhBBShzIPuyOEEELYSJlrM3XYCSGEkDqU+Sw+IYQQwkbKXJupw04IIYTUocxn8QkhhBA2UubazMpJ58zNzTFlyhTR4xUrVsDc3BzZ2dkMpnp/cDgcLFzwOW7eiEFZyX08vJ+IHzd9A03N1kxHE6GMsmN7PoBdGb39vfDVrv9hX+xeRDw+g/2X9tXbtr+jDRZsCMDWk1vwz93jiHh8BlYOfRUXtg42vYdvqqiqwUe/HIf114ew8WSixP5/b2bCb08EHNYFY/C3RzD9j39x8U4OA0lJY1BtJoQQQhSPlR3299m2bdtgbm6O+Ph4xjL8/NMa/PzTGqSn38HCRV8jNPQkAgJm4HjYfnA4HMZy1UUZ3/98ALsyzlgxHdZD+iEvMw+lL0obbDvcfThGeo8El8tF1r3HCkooHZvewzftOncdRS9fSd0XeOEWvjgSi6oaHvxd+mGusxUqqmqw4K/zOJX6UMFJxQkEfJn+kJaHDbWZEEJI/ZS5NreIIfFLlizB559/DmNjY6ajtHiWlj0R4D8Dx8JOwct7lmj7w0dZ2PLrenh7f4zg4HAGE1JGZcgHsC/j1CHT8STrCQDgt8hd0GjgCvW+H/Zj64ptqK6qxsTZn6B7nw8UFVMM297DutJzC3HocgYWjbTBz2dTxPY9L6vAznPX0d1IDwdnj4aaivDcsY+DOXx3nsamU0lwMjeFtoYaE9HBV+Jhd01BtVk+uFwuNn73P/j5eUFDQx3/RcZg7rwv8fx5EdPRALA/H8D+jGzPBzCXkcvlYsbK6RjhOQKt1NWQciEFW1ZsQ0lRiURbg/YGmP+dP7pZdoOxqTE2LfgB58KixdqYW5vj81UzYWZhhuqqaiRfSMHuNb+99cR8c/0uTLyHPD4fW/69hn+uPkBlDQ+DunfAqvH20NfSkNp+f2waQhLuovDlKxhot8bkwRbwHtgTAJBZUIJt/11D6uMCvKysRns9LUwebAEP2+5y/R2kUeba3CKusBsZGeGDDz6AmhozB2/vEx/vCeByudi69Q+x7X/8eRgvX5bjU18PhpK9Rhllx/Z8APsy1nbWG+P5k+eorqqWY5rGYdt7WIvH52Pd8XgM7t4BzpadJPanZhWgmseHWz8zUWcdANRUuBhj1RUlFVU4n8HcyAWBQCDTH2VBtVk+vvwiAOPGj8LgD8eii5ktAGB/4FaGU73G9nwA+zOyPR/AXEZvfy8MGumAheMX4VN74S04X2xZJrWtgM9H8oUUfD//BzzLfSaxn8vlYt2+tbiVmAYvax987jwLBsYGmLdurlx/h1pMvYd7L6ThfEY2Ds4ehYjl7gCAVaGXpLY9n56NXeeuY4PnEFz62hvrPxmEzREpuHwvDwBQUlEF227GODRnNOJWeeHrj+3xS0QKom5lyf33eJMy12bGOux8Ph+BgYEYNWoU+vbti+HDh+OXX35BVVWVRNv67pOLjIyEn58fhgwZgj59+sDR0RFTpkxBUFCQxHMkJyfDz88PNjY2sLOzw+zZs3Hv3j2pz93Q0Lj4+HiYm5tj27ZtYtvT09OxaNEiODs7o0+fPnBwcICHhwc2bNgAHo8HAHB2dsb27dsBAH5+fjA3N4e5uTmcnZ2b/ga+I9sB/cDj8ZCQeE1se2VlJVJTb8HW1lphWepDGWXH9nxAy8jIdmx9D/+6lIGHz4qxYqyd1P1VNcLvRA01FYl9Gq2EA7+uPy6QX0BSL6rNzNTmuj6b+Sl+/HEHHj7MQklJKVas/A6jRzujc2cTRvK8ie35APZnZHs+gLmMYyaNxtGdIXiS9QTlpeX4Y8Ne2A23g5GJkUTbwqdF+Gf/SaQlpYHHkxzyrKWriTYGevj36H/g1fBQ+qIMF05eQDdLM7n+DrWYeg+PJd3FdEdLmLbVgY5GKywaZYO4u3nIfVEm0fZxYSl6tteHVad2AIB+nQ3Rs70+7jwRjgLo26kdfAaaw0hXExwOBzZdjDCke0ckPXoq19+BiGNsSPyaNWtw5MgRdOjQAT4+PgCA8PBw3Llzp1E/HxQUhDVr1sDQ0BDOzs7Q19fH8+fPkZGRgfDwcPj6+oraXr58GZ9//jkAYNSoUTAxMUFqaip8fHxgYWEh8++SkZEBLy8vcLlcODs7w9TUFGVlZcjMzMThw4exdOlSqKiowM/PD1FRUUhISIC7uztMTIT/YHV0dGTO0FgdOhqjoKBQ6sFXTu4TDB5sBzU1NVRXM3f1kDLKju35gJaRke3Y+B7mFJVh17nrmD2sL0z0tZFTJHmA8IGRHgAg4UE+Jg0S/w5OfJAPAMgvLpd/2Hoo87A7qs3M1OZaenq66NLFFClXb4i2PXiQieLiElhZWSIri9lJGdmeD2B/RrbnA5jLqKWrBWNTY9y9cU+0LS8zDy9LXqKbpRme5jStk1j6ogynDp7CmEmjsf/HA9DU0cSw8U6IOyv9anNzYuo9LKmoQl5xOXp1bCva1qmtDrTV1XAn7wU6ttEWaz+qbxeEp9zH1cyn6NfJENeyniGzoARDenSU+vwVVTW4nl2AWcP6yCV/Q5S5NjPSYU9MTMSRI0dgZmaGv//+G9rawg/P/Pnz4enp2ajnCAkJgZqaGo4fPw4DAwOxfYWFhaL/5/P5WLVqFaqrq7F//344ODiI9m3evBm7d++W+fcJDw9HVVUVduzYAVdXV7F9L168QKtWrQAA06ZNQ2lpqeigYODAgTK/dlNptm6NykrJg3sAePWqUthGszWKi5nrJFFG2bE9H9AyMrIdG9/D9ScSYKqvjclDetXbpkd7fTh80B7nM7KxOSIFH9sI5wA4cfUB4u7mAgBeVfMUklealj507l1RbWauNtfS0RG+58XF4vfrvnhRAl1dxZ9AeBPb8wHsz8j2fABzGTW1hHPHvCx9Kba9rKQMmtqa7/ScF07FYuH38+HxmTtUVFVwNfYajmw/KnPWt2HqPSz//9v1dDRaiefRUMPLSsljgbZaGnDt3RmfB0aJat/yMQPQ3biNRFsen49VoZfQXk8TY627ySF9w5S1NgMMDYkPCwsDAMybN090QAAAurq6mDu38feVqKqqQlVV8pxD27avzyolJycjOzsbH374odgBAQDMnj0benp6TY1fLw0Nyckc2rRpw/hMzXWVV1RAXb2V1H0aGurCNuUViowkgTLKju35gJaRke3Y9h6euvYQV+7n4avx9mL3pkvzg/eHcLHshANx6fDYdhIe207i35uZWPn/w+i11Jm7L5ovEMj0p6Wi2sy80lLhiBQ9PV2x7W3a6KKkRP6TZL0N2/MB7M/I9nwAcxnLXwrrlZaOlth2bV1tlJc1fdRVx64dsf7AOhzeGoRxPT6Gu+UnyMvMw3d/fdsseRvC1Huo2UpYO0tfiZ/ML31VLbWu7jl/E2euP8KReW5IWuOLI/Pc8NelDIQl3xNrV83jY8XRODwrrcDWycPeWuPlQVlrM8BQhz0jIwMAYGtrK7HPzk76PY9vGjduHCoqKvDRRx9hw4YNiIyMFDt7Xys9PR0A0L9/f4l9mpqaMDc3b0p0qdzc3KCiogJ/f3988cUXCA8PR1aW4idjaIy83Hy0a9dWdGWhLpOO7fHs2XPGhyBTRtmxPR/QMjKyHZvew6oaHn46m4wPe3SEgXZrZD0vRdbzUuS9EF4pKXtVhaznpSipEB5E6LZWx8++QxH5hQf2zhyB4LljcHLxxzDUFV5h6WqoW+9ryZtAxv9aKqrNzCsuLkFmZjZsrF8PNzUz6ww9PV3cuJHOYDIhtucD2J+R7fkA5jK+LHmJ/Ox8dO/7egby9p3bQ0tXCw/Tm77cZzfLbigrLsN/IZHg1fBQXlqO4/tOoO/AvtDS1Xr7E8iAqfdQt3UrdNDTREbu65noswtLUVZZjR7tJa+ap+c+h3OvTvjASA8cDgfdjdtgeC9TxGS8HrJfWc3DksMxKHz5CrunOktcvVcUZa3NAEMd9tJS4Zmldu3aSeyTtk2a6dOnY9OmTejYsSMOHjwIf39/DB48GNOmTRMdCABAWZnwDNebQ/Oa+noNsbKywqFDh+Dg4ICIiAh8+eWXGDFiBNzc3HD27FmZn785JSWnQkVFBfZ24pNRqauro1+/3khOTmUo2WuUUXZszwe0jIxsx6b38FU1D0UvK3HxTi7G/3pC9OezvZEAgFOpjzD+1xMSZ+0NtFujf1cjWHRsCy6Xg9g7wiHxjj2l3z9H5IdqMzv88echLF/uj65dO0FHRxsbN3yFiIhoZGZmv/2HFYDt+QD2Z2R7PoC5jGcOn4XXXE8YdzKGprYmZq6cgaTzScjPln7/upq6GtTU1cDhAKpqqlBTVwP3/6/+3r1xF1o6WnB2Hw4ul4vWWq0xfuo45P7/ffHyxtR76GHbA4EXbyGnqAxlr6rx67/XMLh7B5joa0u0te5siOj0x8h8Lhy6/+BpMaLTs2H5//fAl1dWw/9gNKp5fOzwGw5NBke/KTNGOuy1E7kUFEjOAixtW30mTJiAo0ePIj4+Hnv27MHEiRMRHx+PGTNmoKhIeGapdljf8+fPpT6HtNerHSZXO4NsXbUHNG+ysbHBb7/9hsTERAQFBWHevHl49uwZFi1ahISEhEb/TvJ2NOQE+Hw+Fiz4TGz7ZzMnQUtLE4eDwxhK9hpllB3b8wEtIyPbsek9bN1KFT96O0r8+d844ZXZIT064EdvRwyzMK33OW7lPEdY8j0M6GoEmy6SMwIrirIuHUO1mR02/bAdp07+hyuXTiPrUTJUVLjwmzaf6VgibM8HsD8j2/MBzGU8suMo4iPjse3kFhxKPAiuChebFvwIABg+YTjCM46JtT957wRO3jsBY1NjLP15CU7eO4FJC4STW+Y/zse3s9djwswJCLl+BAcu74OxqRHWzFwr998DYO49nDHUEk4Wpvh091mM/PEY+HwBvps4GABwKvUhBn17RNR26oeWcLbshDn7zmHQt0cw78A5DO9liulDewMAItMeI+lhPq5lPcPw70Mx6NsjGPTtEaw/Iblah7wpa20GGJp0zsLCArdu3UJSUhLGjx8vti8xMbHJz6erqwsnJyc4OTmBz+cjNDQUycnJcHV1Ra9ewomPUlJSJH6uvLwct2/flthee+9cfn6+xL6bN282mKVVq1bo378/+vfvjy5duuDLL79EZGQk7O3tAQjXhASEE+4w4ebNDOzctQ8B/jMQcvR3nDlzDr0seiAgYAZiYi4hKIj5ThJlfP/zsTGji4czjEyFnUS9tnpQVVOF7wLhLNlPs58i6tg5UVszi65wGCm877a3raXw5z9xQW97YYE7HngC5aXyn+WcTe+hmgoXI/p0ltheO0u8aVsdsf07IlOR9bwUfUwNoK2hhvTcQpy4+gBGOpqiAwumKOtMtFSbmavNdfH5fHyx4lt8sUL+99m+C7bnA9ifke35AOYy8vl8/L7+D/y+/g+JfdHh0YgOjxbbNqrTmAafL+FcIhLONf37qzkw9R6qcLlYMro/loyWvOXoo35m+Kjf62XtVFW4WDjSBgtH2kh9rvE23TDeRvETzEmjrLUZYKjDPmHCBISGhmLXrl1wdnYWnWkvKSnBrl27GvUcV65cwcCBAyUmjam9V651a+F9kP3794eJiQliY2Nx5coVscltfvvtNxQXF0s8d9++fQEAx44dw/jx46GiIlwr+N69ezhw4IBE+5SUFFhaWkpMbFN75aA2CyCc6AYA8vLyGvV7ysOSpauRmZmNzz77FG5jXFBQUIgdOwKxeu2PrDkDRRnf/3wAuzKO8hmFfoOsxLZNWz4VAJB6+bpYh7173+6ifbVG+4wS/f+5Y+cU0mEH2PUeNkWvjm0R/+AJLt/Pw6vqGrTX04LPQHPMGNobuq2ZuT+uFpvfN3mi2sxsbSaEEFI/Za3NAMARMPTbf/311zh69Cg6dOiAkSNHQiAQICIiApaWloiOjoa9vT0OHjwIAFixYgXCwsIQFRUFU1PhcEpbW1toamrC2toaJiYmEAgESEpKwo0bN2BlZYWgoCDRLLWxsbGYM2cOAGD06NGitV5v3rwJCwsLJCYmIjo6Gh07vr5n0tfXFykpKejduzcGDhyI/Px8REVFYdiwYTh79iwCAgIwf75wWMu8efNw5coV2NrawtTUFJqamrh37x4uXLgAHR0dhIaGinLfu3cPY8eORbt27TBu3Djo6OhAV1cXkydPbvR7p9rKRPa/AELecy7GVm9vxLCo/OtMR2hQ6V+zmY7wVq29vmn252yn21Omny8oadya5WxEtZlqMyHyRLVZdlSb301Lrs2MXGEHgLVr18LMzAxHjhzB4cOHYWhoiI8//hjz588XnUVvyNKlS3Hx4kXcunULMTExUFdXh4mJCZYvXw5fX1+xJWU+/PBD7Nu3D1u2bEFkZCRUVVUxYMAABAUF4aeffgIAsSVsAGDXrl3YtGkToqOj8ddff6F79+7YuHEjDAwMJCarmTRpEvT09HDt2jUkJyeDz+fD2NgYkyZNwowZM8QONmqfZ+/evfjrr79QVVUFExOTJh0UEEIIkZ+WvvyLLKg2U20mhBA2UubazNgVdjbg8XhwdXVFVVUV4uLimI7TaHQWn5C3o7P4slPWs/j62t3f3qgBRWX33t6I1ItqMyHvL6rNsqPa/G5acm1mZJZ4RauoqBAtIVNLIBBg165dyM3NhaurK0PJCCGEsA0fApn+kMah2kwIIaSxlLk2MzYkXpEeP34MLy8vDBkyBJ07d8arV6+QmpqKW7duwdjYGAEBAUxHJIQQwhJKPPBMoag2E0IIaSxlrs1K0WE3NDSEm5sbEhMTcenSJVRXV8PIyAiTJk3C3LlzYWhoyHREQgghRKlQbSaEEELeTik67Pr6+tiwYQPTMQghhLQAyjyxjSJRbSaEENJYylyblaLDTgghhDSWoIXf60YIIYS8b5S5NlOHnRBCCKlDmc/iE0IIIWykzLWZOuyEEEJIHUxMbJOWloatW7ciJSUFr169Qrdu3eDj4wNvb29wOByF5yGEEELYpKXWZoFAgODgYBw5cgQPHjyAhoYGBgwYgAULFqBXr16Neg6lWNaNEEIIYaukpCR4e3sjNjYWTk5OmDJlCmpqarB69WqsW7eO6XiEEEKI0mmu2rx27VqsWbMGJSUlmDRpEkaNGoX4+Hh4eXkhKSmpUc9BV9gJIYSQOhR5n1xNTQ2++uorVFVVYc+ePXBycgIALFy4ENOnT8fhw4fx0UcfwdbWVmGZCCGEELZpibU5MTERQUFB6Nq1K/7++2/o6OgAACZNmgQvLy989dVXOH36NFRUVBp8HrrCTgghhNQhEAhk+tMU8fHxePToEQYOHCg6IACAVq1aYeHChQCA4ODgZv39CCGEkJamJdbm2jZz584VddYBoFevXhg7diwePXqE+Pj4tz4PddgJIYSQOhR5UJCQkAAA+PDDDyX2DRgwAJqamqI2hBBCiLJqibW5ts2QIUMk9tU+d2M67DQknhBCCGlGJSUlKCkpkdiuq6sLXV1dsW0PHz4EAHTp0kWivYqKCkxNTXHnzh1UVFSgdevW8glMCCGEvOcUXZvLy8vx9OlTaGpqwtDQUGJ/7XNnZma+NTt12FugmqocpiMQQsh7q1rG79ht27Zh+/btEtsDAgIwf/58sW1lZWUAIDZUri4tLS0AQGlpKXXYWY5qMyGEyE9Lq82lpaUNPoe2trZYu4ZQh50QQghpRlOnToW7u7vE9jfP4BNCCCFEMVpybaYOOyGEENKMpA2vq8/bzrC/fPkSQP1n6AkhhBDydoquzbX76nuOt13Fr4smnSOEEEIYYmZmBkD6PWw8Hg/Z2dkwNjam4fCEEEKIgjRHbdbU1ISRkRHKy8vx7Nkzif21zy3tPvk3UYedEEIIYYi9vT0AIDY2VmJfcnIyysvLRW0IIYQQIn/NVZtr28TFxUnsq33ugQMHvvV5qMNOCCGEMGTgwIHo2rUr4uPjERMTI9peVVWFLVu2AAC8vb2ZikcIIYQonabW5sLCQty/fx+FhYViz1PbZteuXWJD49PT03Hy5El07dq1UR12jqCpC9MRQgghpNkkJiZixowZEAgEcHNzg6GhIWJiYnD37l34+vpizZo1TEckhBBClEpTanPtDPTSZpxfs2YNgoKCYGJigpEjR+Lly5c4deoUqqurERgYCFtb27dmoQ47IYQQwrBbt25h27ZtSE5ORmVlJbp27QpfX1/4+PiAw+EwHY8QQghROo2tzQ112AUCAYKCghAcHIxHjx5BXV0dAwYMwIIFC2BpadmoHNRhJ4QQQgghhBBCWIjuYSeEEEIIIYQQQliIOuyEEEIIIYQQQggLUYedEEIIIYQQQghhIeqwE0IIIYQQQgghLKTKdABCSPOLiIiAq6srVFRUmI7SYlVXV0NNTY3pGACES4u8Kzs7u2ZM0jQuLi6YOnUq/Pz86m1z6NAh7N27F1FRUQpMRgghike1WXZUm2VHtbnloQ77e64lfJmEh4e/889OmDChGZM0Xk5ODvbv34+MjAzk5+ejpqZGog2Hw0FkZCQD6YCFCxeiXbt28PDwgKenJzp16sRIjlq5ubnv/LMdO3ZsxiSN5+joCHd3d3h6eqJbt26MZKg1ZcqUd17aKz09vZnTNF5OTg5KSkoabFNSUiLT54OQlohqs3xQbW4aqs2yodpMFIU67O+5lvBlsmLFiiZnFAgE4HA4jBwUXLp0CXPmzEFVVRVUVVVhYGAg9Ww5kysmTp8+HcePH8eePXvwxx9/wMHBAV5eXnB1dYWqquL/2Ts7O7/T55DD4SAtLU0Oid5OVVUVgYGB2LdvH2xtbeHl5YVRo0ahVatWCs/i7+//3q7FXVpaysh7SgiTqDY3P6rNTUe1WTZUm4mi0Drs77lt27ZJfJmkpqbi4sWL6NKlC/r374927dqhoKAAKSkpyMzMhKOjI/r164eAgACFZAwLC3vnn3V3d2/GJI1/zXv37mHjxo1wc3MDl8vOqSCqq6vx33//ISQkBFeuXAEAtG3bFu7u7vDy8kLnzp0VlkXagV92djYSExOho6ODXr16iT6H6enpKC0tha2tLTp16oSNGzcqLGddPB4P0dHROHLkCOLi4iAQCKCrq4sJEybAy8sLH3zwASO52K7ulcMpU6bA3d0dHh4eEu14PB6ePHmCn3/+GW3btsXx48cVGZMQRlFtbn5Um5uOarPyoNrcslGHXclcvXoVfn5+WLlyJXx9fcW+qAUCAQ4fPoxNmzZh//79sLGxYTApe1lZWeGjjz5irFi9i8ePHyMkJATHjh1DQUEBOBwO7O3t4e3tjREjRij8frD79+/D19cXnp6emDt3LrS1tUX7Xr58ie3btyM0NBTBwcGMD3kDgLy8PNH79+TJE3A4HNjY2MDb2xtjxoyhs9B1WFhYNPqKQ+3VuI0bNzI2hJYQNqDaLDuqzbKj2vz+otrcslGHXclMnToVWlpa2LlzZ71t5s6di4qKCuzbt09xwVqQYcOGYcSIEfjqq6+YjtJkPB4PUVFR+O677/D06VMAQJs2beDp6Ylp06ahbdu2CskxZ84cVFRUYP/+/fW28fPzg7a2doOfVUXj8/mIiYlBSEgIYmJiwOfzoauri/Hjx8PHx4fO7OP1lUOBQIAdO3bA3t4e9vb2Eu24XC709PTg4OBA7xtRelSbZUe1WXZUm99fVJtbNrqHXcncuHEDU6ZMabBNz549cfDgQQUlanlGjRqFuLg41NTUMHLP2bsqKChAaGgo/v77b+Tn54PL5WLQoEG4e/cu9uzZgyNHjmDHjh2wtbWVe5aUlBT4+vo22MbGxgZBQUFyz9IUXC4XlpaWsLS0xPXr11FQUIDi4mIcPHgQhw4dgpubG9asWSN2VUIeXFxcGtWOicmV5s+fL/r/sLAwuLq6NjgTLSGEanNzoNosO6rNsqHaTOSl5XyjkWahoqKC+/fvN9jm3r17jC858uTJE+zatQuXLl1Cfn4+qqurJdowNenJokWLkJKSgqVLl2LFihXo0KGDwjM0xYULF3D06FGcP38eNTU1aNeuHebOnQsvLy906NABPB4PJ06cwPr167Fx40aEhobKPVN1dfVbZx/Nzs6W+vfOBD6fj+joaISEhODixYvg8/no0KEDFi1aBHd3d2RkZOCPP/7AyZMnAQA//fSTXPPUNzCqrKxMNPOrkZER4wet586dY/T1CWkpqDbLjmqz7Kg2y4ZqM5EX6rArGTs7O5w7dw4hISHw9PSU2B8SEoJz587B2dmZgXRCOTk58PT0xIsXL9C9e3dUVVWhY8eOaNWqFbKzs1FTU4NevXrJ/UxpfVq3bo1169Zh6tSpcHZ2hq6urtQsTC4dk5+fj7///huhoaHIy8sDADg4OMDHx0diDVgVFRW4u7vjzp07+OuvvxSSz8rKCmfOnIG7uzsGDx4ssT82NhYRERGMrlMKCD+LtffHPXv2DBwOB0OHDoWPjw+cnJxE94MZGxvDyckJ/v7+uHDhgtxzNVRss7Ky8N1336G0tBR79+6Ve5aGCAQCVFdXS9xHmJSUhMjISGhoaMDLy4ux5YEIYQuqzbKj2iw7qs2yodpM5IXuYVcyDx48gLe3N8rKytC1a1eJmWgfPXoEHR0dRicUWblyJU6cOIE//vgDgwYNgoWFBQICAhAQEID8/HysXr0ajx49QnBwMNq0aaPwfFeuXMHs2bNRWVnZ4NIxAHNnMS0tLSEQCKCnpwd3d3f4+PigS5cuDf7Mnj178MsvvyAjI0Pu+a5duwY/Pz9UV1dj8ODBGDBggOhzmJycjEuXLkFNTQ0HDx5Ev3795J5HmpkzZ+Ly5cvg8/lo164dJk6cCG9v7wav2vz222/49ddfGV1fFQCqqqowbtw4ODs748svv2Qsx8aNvjlkhAAAIABJREFUG3H48GHExcVBV1cXABAREYHFixeDz+cDAPT19REWFob27dszlpMQplFtlh3VZtlRbZYvqs3knQmI0rl7965gypQpAnNzc4k/U6ZMEdy7d4/RfE5OToLZs2eLHpubmwu2bdsmelxRUSFwcXERrF69moF0AoGnp6egT58+gn/++UfA4/EYyfA2Pj4+guPHjwsqKyuZjlKvK1euCFxdXUWfPQsLC9H/jxgxQnDlyhVG85mbmwumTZsmOHv2rKCmpqZRP3P79m3BsWPH5JyscdauXSsYOnQooxnc3d0FM2bMENvm5uYmsLe3F4SHhwv++OMPgaWlpWDDhg0MJSSEPag2y4Zqc/Og2ixfVJvJu6Ah8Uqoe/fuOHDgAJ48eSJaV7N2vU02nEkrKChA9+7dRY9VVFRQWVkpeqyhoYEhQ4YgOjoaa9asUXi+O3fuYOzYsRg7dqzCX7ux2DYhjDQDBw7Ev//+i+TkZGRkZIg+hxYWFhgwYECjlx+Rl4iIiLde+XhTz5490bNnTzklaho+n4/CwkJGM+Tl5YktQfX48WPcv38f/v7++PjjjwEI14a9ePEiVq5cyVRMQliBarNsqDY3D6rN8kW1mbwL6rArsfbt27PiIOBNenp6ePXqlehxmzZtJCZB0dDQwIsXLxQdDQDqvS+OrXg8Hh4+fCgqumZmZoxPXFSLw+HA1tZWIbPfNpWXlxdcXFywYcMGpqM02aNHj3DmzBl06tSJ0RxlZWVi/1aSk5PB4XDg6Ogo2tajRw8kJCQwEY8QVqLa/G6oNjcfqs3yQbWZvCvqsCupqqoqXL58GQ8fPsTLly/h7+8PAKisrERZWRn09fXB5XIZydapUyfk5OSIHltaWuLSpUsoKiqCvr4+KioqcO7cOZiYmDCSb8SIEbh8+TJ4PB5riqs0RUVF2Lx5M/755x+xgywNDQ2MGzcOixcvhr6+PoMJ2a2mpgYGBgZMx5CqvjPePB4P+fn5SE5ORk1NDZYsWaLgZOIMDQ2RnZ0tenz58mVoaGigd+/eom3l5eWs/ndEiCJRbX53VJuVA9Vm2VFtbnmow66Ezp8/j1WrVuH58+cQCATgcDiig4KMjAz4+Pjghx9+wLhx4xjJN2TIEOzduxcVFRVo3bo1fHx8RMN0rK2tcevWLeTm5mLp0qWM5Fu8eDFmzJiBJUuWsHbpmKdPn8LX1xc5OTnQ1dVF3759RRPHZGRk4OjRo7h06RKCgoJgaGjIWM7U1FTExcUhPz8fVVVVEvs5HA5jZ9F79eqFBw8eMPLabxMWFtbgfjMzM8yYMUPqbNOKZGVlhXPnziE6Ohrq6uqIiIiAg4MD1NTURG2ys7NhZGTEYEpC2IFqs2yoNjcfqs3vhmozkReaJV7JpKam4tNPP0W7du0wc+ZMpKam4tSpU2IzZ44cORIWFhbYunUrIxmfPn2KpKQkODg4oG3btgCAffv2YceOHSgtLYWGhgYmT56MJUuWMHKlwcXFBdXV1Xj27BmA+ofhMbl0zJdffonjx49j1qxZmDNnDjQ1NUX7ysvLsWvXLvz+++9wd3fHxo0bFZ6Pz+dj2bJlOHPmjOjAtO5XUe1jDofD2KyuMTExmDdvHvbs2YMhQ4YwkqE+da9y1cXlcqGjo8OaYaEZGRnw8vISrdnL5XJx+PBh0ezClZWVGDx4MEaNGtUihzcS0lyoNsuOarPsqDbLhmozkRe6wq5kdu7cCS0tLfz9999o164diouLJdr06dMHN2/eZCCdkJGREdzc3MS2TZs2DVOmTEFRUREMDAwYnfREIBBAVVVV7Oy9tPNeTJ4Li4mJgYODg9RhV5qamli6dClSU1MRHR3NQDrhQd7p06fh7u6OyZMn45NPPsHUqVMxZswYJCYm4vfff8eQIUOwbNkyRvIBwmGLH374IT7//HO4urqiT58+MDQ0lPrZmzBhglyzrFy5Eq6urnBxcQEgPGhqCfdrWlhY4OjRozh+/DgAYMyYMbCyshLtT0tLg4ODA6sniSJEEag2y45qs+yoNjcN1WaiKNRhVzJXr17FiBEj0K5du3rbdOjQgbFiAQABAQHo06cP5syZI7ZdRUWlwdyKwtT6rU3x6tWrt66RamNjgxs3bigokbgTJ07ggw8+ELuCoKOjA2tra1hbW8PR0RFeXl4YNGgQvLy8GMm4YsUK0dWEf//9F//++y8AiB0U1F5pkPdBQVhYGExMTEQHBS4uLvD390dAQIBcX7c5WFhYwMLCQuo+Gxsb7NixQ8GJCGEfqs2yo9osO6rNTUO1mSgKddiVzKtXr6Crq9tgm5cvXyoojXQXLlyAmZkZoxlaup49e9Y7NKtWdnY2evTooaBE4jIzM+Hu7i62jcfjif7fwsICzs7OCAoKYuyggInhiPXR0dER+3dJdzIR8n6h2qwcqDbLjmozUUbUYVcyHTt2REZGRoNtbty4ga5duyomkBRdunRBfn4+Y6//Ppg1axYWL14MDw8PDB48WGJ/bGwsIiIiGLsXksPhQEtLS/RYU1MTRUVFYm1MTU0RExOj6Ggibx60MKlbt244deoUrKysRFeycnJykJiY+NaftbOzk3e8em3fvr1R7epOrkWIMqLarByoNsuOarPsqDa3PNRhVzLOzs4IDAxEdHQ0hg8fLrH/2LFjSEtLY3Q4z8SJE7Fz507k5eWxYpbXxn6xvYnJL7qysjIMGTIEM2fOxODBgzFgwADRTLTJycm4dOkShg0bhpKSEoSHh4v9rLyHkAHCdYafPHkiety1a1ekpqaKtbl9+7bYgYMyqx1iV/e+x/DwcIm/O2mYmhgIaPjfTu3wxTdnwyZEGVFtbjqqzc2PanPTUG0mikKzxCuZFy9ewN3dHU+fPoWbmxtevHiB2NhYLF++HFevXkVkZCRMTU0RFhbG2KQZ2dnZWL9+PdLT0/H555+Llj2RNqFIx44d5Z6nvnt83obJWVQtLCwkZneVRto9X4rIvGrVKsTHx+O///4DAOzYsQPbtm2Dp6cnXFxckJiYiL1798LNzQ0///yz3PNIk5ub+9Y2XC4X2traCvm38vjxY1y5cgX5+fnYvn077O3tYW9v/9afY/IAPyEhQer20tJS3LhxAwcPHsTQoUPh6+vbqN+FkPcV1eamo9rc/Kg2Nx3VZqII1GFXQtnZ2fjiiy+QkpIisc/GxgY///yzQoptfeoWtIZmnOVwOEhLS5N7nvq+2BqDqS+6t60F2hBFDDeLj4/H77//jrVr18LExATl5eWYMmUKbt26Jfq779SpEw4cOMDYlZzaz2FjGBgYYOTIkfD394eBgYGckwmzBQQEtIiJbRpy7949TJw4EZs2bcKoUaOYjkMIo6g2Nw3V5uZHtVk2VJuJvFCHXYllZGTg6tWrKC4uho6ODqysrNC3b1+mY4lmAG0MNk0+QmRTU1ODyMhIZGVlwdTUFMOGDRNbo1bRVqxYIboXTU9PD+bm5qKhi7dv30ZxcTHs7OygpaWFO3fuIDc3Fx06dEBISIjcZ0zev38/hg0bhi5dusj1dRRh6dKlePjwIY4dO8Z0FEJYgWozYROqzY1HtZnIC3XYCSGsUVxcjOTkZLRu3RoDBw4El8tlLMv9+/fh4+ODyZMnY9asWWjdurVoX0VFBXbv3o2goCAEBQXBzMwMO3fuxPbt2zF58mSsWrVKrtl69eoFT09PrFu3Tq6vowibNm3C4cOHJe6TJIQQwg5UmxuHajORF+qwEwDs+jLOzc2Frq5ug/celZWVoaSkhNHhgYBwuZOioiJUVVVJ3c90vpycHJw8eRJpaWkoKyuDtrY2LC0tMXbsWJiYmDCWKzg4GOHh4di9ezfatGkDQHhVaebMmSgsLAQA9OvXD4GBgWLFWJHmzJmDyspKBAYG1ttm+vTp0NDQwK5duwAAHh4eKCkpQWRkpFyzDRo0CBMmTMCXX34p19eRNz6fD09PT+Tl5eHSpUtMxyGEdag2vxuqze+GarNsqDYTeaFZ4pVMS/gydnFxEc28WZ+DBw9i69atjE0ck5aWhl9++QUJCQmorq6W2kZR9/HVZ+/evdi8eTNqamrEJriJiIjA9u3bsWTJEkyfPp2RbKdOnQKPxxN9BgHh2dyioiJ4eHjg+fPniImJQXBwMGMZU1JS4Ovr22AbKysrBAUFiR5bW1sjJCRE3tFgZ2eHq1evyv11ZFXf0jY1NTXIz89HWFgY0tLS4OHhoeBkhLAL1ebmQbVZNlSbZUO1mcgLddiVTEv4Mm7MoA8mB4bcvn0bkyZNAofDwZAhQxAdHQ0LCwsYGBggLS0NRUVFsLe3Z/Qs+fHjx/HDDz9AX18ffn5+sLW1hYGBAZ4/f47ExEQcOHAAP/zwAwwMDDB+/HiF58vMzISTk5PocWFhIa5cuSI2lMzT0xP//PMPY5/D6upq5OXlNdgmNzdX7KBQXV0dampq8o6GxYsXw8vLC1u2bMG8efMU8prvYsqUKQ3e8yoQCNC/f/8WfzWCEFlRbZYd1WbZUW2WDdVmIi/UYVcyLeHLuDGePHnC2DqgO3fuBJ/PR2hoKHr06AELCwu4uroiICAAFRUV2LBhA6KiohiddCcwMBD6+voIDw+HsbGxaHu3bt1gZ2cHDw8PTJgwAYGBgYwcFBQVFaFt27aix7WzIru6uoq22draMjrZSd++fXHmzBl4enrCzs5OYn98fDzOnj2L/v37i7Y9fvwYhoaGcs+2Z88e9OjRA7t370ZISAgsLCykvi6Hw8GGDRvknqc+/v7+Ug8KOBwO9PT00LdvX/Tr14+BZISwC9Vm2VFtlh3VZtlQbSbyQh12JcPWL+Pt27eLPU5ISJDYBgjvq8nLy8Pp06cZ+zJJTk7G8OHD0aNHD4l9rVu3xpo1a3Dt2jVs3rwZP/30EwMJgQcPHmDixIliBwR1tW/fHqNHj5ZpiRlZ6OnpoaioSPQ4MTERXC5XrMACqPf+Q0VYuHAhpk2bBj8/Pzg6OsLGxkZ0JSQlJQWxsbFQVVXFggULAAjv3YyLi8NHH30k92x1/94KCgoQGxsrtR3TBwXz589n7LUJaUmoNsuOarPsqDbLhmozkRfqsCsZtn4Z1z0A4HA4SEhIaHCNVSMjIyxbtkwR0SS8ePECnTt3Fj1WVVVFeXm56LGKigoGDhyIs2fPMhEPAKCrq/vW+xw1NTWho6OjoETiunXrhujoaBQVFUFFRQWnTp1C3759xSYzysnJkfsSLA0ZMGAA9uzZg1WrVuHChQu4cOGCaB1aADAxMcG6deswYMAAAMLPwdGjRxVyFj8qKkrur0EIURyqzbKj2iw7qs2yodpM5IU67EqGrV/GBw4cACC8b2bq1Klwd3eHu7u7RDsul4s2bdqgW7dujM2W27ZtW5SVlYk9zs7OFmsjEAjw8uVLRUcTcXZ2RnR0NBYvXgxVVcl/5jU1NTh//jxcXFwYSAf4+fkhICAATk5OUFFRwatXr7B8+XKxNqmpqYyvPTxo0CBERkYiOTkZ6enpotl8e/Xqhf79+4t9BjU0NKRe2ZEHJu/BbIrc3Ny3tuFyudDW1m5w5mlC3ndUm2VHtVl2VJtlQ7WZyAt12JUMW7+M7e3tRf/v7u4OV1dXsW1s0rVrV2RlZYke9+vXD3FxccjKykLnzp1RUFCAiIgIsTP9irZ8+XJMmzYNM2fOxOLFi2FtbS3ad/XqVWzevBmampqMXQlxdXXF6tWrRbO2jh8/Hh9//LFof3x8PMrLy+Ho6MhIvro4HA5sbW1ha2vLdJQWx9nZucGJbeoyMDDAyJEj4e/vDwMDAzknI4RdqDbLjmqz7Kg2KweqzS0PrcOuhIKCgsS+jKdNmybaFx8fj4CAACxbtgze3t4MJWS333//HVu3bkVsbCz09PSQlJQEPz8/qKuro1u3bsjMzMTLly+xfv16fPLJJ4xkdHFxQXV1NZ49ewZAOBRQX18fRUVF4PF4AABDQ0OJGUw5HI7c1yklzaeqqgrXr1/H06dP6x0qO2HCBAWnem3FihXIyclBYmIi9PT0YG5ujnbt2qGgoAC3b99GcXEx7OzsoKWlhTt37iA3NxcdOnRASEgIo0MuCWEC1WbZUG0mbEG1mTQ36rAT1mnMUJ1aHTt2lGMS6crKynD//n188MEHoqFCERER2LZtG7KysmBiYgI/P7+3rhMqT87Ozu/8s+fOnWvGJC1bUVERQkNDcePGDZSUlIgOqOricDjYv3+/wrMFBQXh119/RUlJidT9AoEAHA6HsfWQAeD+/fvw8fHB5MmTMWvWLLF7NysqKrB7924EBQUhKCgIZmZm2LlzJ7Zv347Jkydj1apVjOUmhEii2iw7qs3Ng2qzbKg2tzzUYSesY2Fh0aihOhwOB2lpaQpIRJTR3bt34efnhxcvXjS4tjAThTcqKgr+/v7o0aMHPvnkE3z//fdwdXWFlZUVEhMTcfHiRYwaNQrDhg2Ter+posyZMweVlZUIDAyst8306dOhoaGBXbt2AQA8PDxQUlJCV5MIYRmqzYQNqDbLjmpzy0P3sL/nas+IGxsbQ0VFhfVnyAHhMCFpBwWlpaVIT09HTk4O7OzsYGpqykA6IDo6GqGhofj666+lLs2Sn5+Pb7/9Fp6enmLr6pKWZdOmTSgqKsLs2bPh6emJDh06QEVFhelYAID9+/dDX18fwcHB0NLSwvfffw8LCwvMmjULs2bNQnh4OL766itMnjyZ0ZwpKSlvvZplZWWFoKAg0WNra2vRsGBC3ldUm5sf1WblQLVZdlSbWx7qsL/naieWOH36NMzMzBo90QSTZ8i///77evfx+Xzs3r0bhw4darCdPB0+fBhPnjypdx1VY2NjZGVlISgoiA4KWrCUlBQMHz4cixcvZjqKhPT0dIwcORJaWlqibXWvNEyYMAHh4eHYuXNng2fQ5a26uhp5eXkNtsnNzUV1dbXosbq6usT9m4S8b6g2Nz+qzcqBarPsqDa3PNRhf8/VnhGvXdOzvjPkLQWXy8W8efMQGxuLH3/8Eb/++qvCM2RkZGDo0KENtrG2tkZMTIyCEtUvNTUVcXFxyM/PlzrxCYfDwYYNGxhIxn4cDgdmZmZMx5CqsrJSbOIXdXV1lJaWirWxtLRk/Gx43759cebMGXh6esLOzk5if3x8PM6ePSu21vTjx48Vsl4uIUyi2tz8qDYrB6rNsqPa3PJQh/099+aZbqbOfDe3fv364dixY4y8dlFR0VtnydTX10dhYaGCEkni8/lYtmwZzpw5I5rgpO5Z3trHdFBQP2tra9y5c4fpGFIZGhqioKBA9Lh9+/a4e/euWJvnz58z3gFYuHAhpk2bBj8/Pzg6OsLGxgYGBgZ4/vw5UlJSEBsbC1VVVSxYsACAcNKouLg4fPTRR4zmJkTeqDY3P6rNyoFqs+yoNrc8XKYDEMU6cOAA/v33X6ZjyOzp06eorKxk5LXbtGmD7OzsBttkZ2eLrpwwYd++fTh9+jQmTJiA0NBQCAQCTJ06FcHBwVi6dCl0dHQwZswYmjykAcuWLUNSUhJOnTrFdBQJvXr1EjsIGDhwIBISEvDPP/+gvLwcMTExOHv2LMzNzRlMCQwYMAB79uxBx44dceHCBWzZsgWrV6/Gli1bcPHiRXTs2BG7d+/GgAEDAACqqqo4evQoY2sQE8IUqs2yo9qsHKg2y45qc8tDV9iVzKZNm+Dn54eRI0cyHeWd8Pl8hIWF4ezZs+jXrx8jGfr374+oqChkZWWhc+fOEvszMzMRGRkJR0dHBtIJnThxAh988AE2btwo2qajowNra2tYW1vD0dERXl5eGDRoELy8vBjLyWZRUVFwcHDAsmXLEBwcjN69e4uWCqqLw+HA399fodmGDx+OdevWITs7G6amppg9ezYiIiLwxRdfiNqoqqpi0aJFCs0lzaBBgxAZGYnk5GSkp6ejrKwM2tra6NWrF/r37w8u9/V5Yw0NDfTo0YPBtIQwg2qz7Kg2Kweqzc2DanPLQsu6KRlnZ2cMHToUa9asYTpKvVxcXKRu5/F4KCgoAI/Hg6qqKgIDA2Fra6vgdMDVq1fx6aefQl9fHwEBAfjwww9hbGyM/Px8XLx4Edu3b0dxcTEOHjwodv+PItnY2MDd3R3ffPMNAOFyPHPnzsXChQtFbRYtWoTMzEyEhYUxkpHtLCwsGtWO6fVUaz1+/BiBgYGi9YZ9fX0b/TsQQphFtVl2VJuVA9VmoozoCruScXFxQUxMDCorK6Gurs50HKnqO4ekqqqKnj17om/fvpgyZQq6d++u4GRCNjY2+Prrr7F+/XqsW7dOYj+Xy8WqVasYOyAAhIWq7iylmpqaKCoqEmtjamrKisl32OrAgQNMR2iSTp3+r737j4m6fvwA/nxLJBA/BE8L1CZSeoUXCsiRzZCgoTYEXThv5oVaanrNSMvN1aw1LSprGQIbldpckIhjI/wxHYRzxs+zC5PzJ/66RiQc6EFIY/f9oy/36YSDDHm/38f7+fjvffdmezIdz/fr/X69X69JjotAuWtvb0ddXR28vb2h1Wqd7uQTKRG7eejYzcrAbh4+7Gb54oBdYTZs2IC6ujqsW7cOmzdvxtSpU6WO1EdZWZnUEQal0+kQHR2N/Px8/PLLL7h9+zb8/PwQEREBnU4n2QVLr0ceeQRNTU2O48mTJ8NkMjmdc+7cOacLB3IWExMjdYR/rb29HZ2dnQgODpY6ipOCggIUFxcjNzcXY8aMAfD3Ss6rVq1yLPwUERGB3bt3w9vbW8qoRJJiN98f7OaRj908dOxm98MBu8KkpKSgu7sbDQ0NSElJwejRoxEUFNRnxUpBELjoySAef/xx2d41jYyMRFVVleM4ISEBX375Jd59910kJCSgpqYGJ0+exIIFCyRM6T6amppcbr8DoN9tUYabzWbDzp07UVpaitbWVqf9mU0mE7KysvDGG28gPDxc9Gy9SktL0dPT47ggAP5+V9dqtWLx4sVoaWlBRUUFCgoKsGLFCslyEkmN3Xz/sJuVg93837Cb3Q8H7Apjt9vh6enZ527f3VPd5LC0QXd3N3766Sc0Njaio6PDsXjInTt3YLPZEBgYyOk6LiQnJ6OpqQkWiwUTJkzAihUrUFZWhsLCQhw4cAB2ux2TJk3iip+D+PHHH5GZmYkrV64MeJ7Y78ndunULOp0Oly5dwpNPPonAwEBcunTJ8f20adNQV1eHkpISSS8Krl69iri4OMdxa2srKisrkZaW5piympaWhpKSEl4UkKKxm5WB3Xx/sJuHht3sfjhgVxh3mNIG/P3H+J133kFLS4tjT9LeiwKz2YylS5fi448/RnJyssRJ5Umr1UKr1TqOfXx88P333+P48eO4du0aJk6ciLlz58LHx0fClPJWW1uL9evXIygoCMuWLcO+ffswa9YshIaGwmg04uLFi4iPj5ekdHft2oVLly7hk08+QXJyMrKysrBr1y7H915eXoiJiUFlZaXo2f7JarUiKCjIcWw0GgEAiYmJjs+io6Ml27eZSC7YzcrAbh46dvPQsZvdDwfsJDsmkwkGgwEqlQpbtmyByWRy2m8zIiICkyZNwrFjx3hRcA8eeOABzJs3T+oYbiMvLw9eXl4oKirC+PHjsW/fPmi1WhgMBtjtdmRnZyMvL89pdV+xHDt2DM8+++yA//9DQkLw888/i5iqr4CAAKcFlWpqajBq1Kg+iz65ms5IRPLBbh4e7OZ7w24eOnaz++GAXcF6enrQ2NjoWJQlNDQUHh4eUsdCdnY2HnroIRw4cAAqlQrt7e19zpk+fTrOnDkjQTr3YbVaUVRUhPr6ety6dQs9PT19zhEEAXv37pUgnfyZTCYkJCRg/Pjxjs96p6P2PlU6ceIEdu7ciezsbFGzNTc3D/qOo5eXFzo6OkRK1L8pU6agvLwcVqsVHh4eKC0thUajcdoz12KxQKVSSZiSSF7YzSMbu3lo2M1Dx252PxywK5DVasXnn3+OkpISdHV1OT738vJCcnIyMjIyEBgYKFm+06dP4/nnnx/wD0VwcDDKy8tFTOVeLly4AL1ej7a2tgHfebx7QSP6n46ODqf3ST09PdHZ2el0zsyZM1FcXCx2NAQEBKC5uXnAc65duyZ52er1ehgMBsTFxcHDwwNdXV146623nM4xmUzQaDQSJSSSD3bzyMduHjp289Cxm90PB+wK09zcDJ1OB4vFAn9/f2g0GqhUKty8eRNmsxn79+/HqVOnkJ+fj3HjxkmSsaurC/7+/gOeI/XdSbnrXe1zzZo1SEtLQ3BwsCye0LgTlUrlNGVs3LhxaGxsdDqno6NDkiljM2bMcNwd7+8C/vr166ioqMD8+fNFz/ZPiYmJ2Lp1KwoLCwEACxcuREpKiuP7qqoqdHZ2Ys6cOVJFJJIFdrMysJuHjt08dOxm98MBu8Ls2LEDFosFq1evxtq1a50WNuns7EROTg7y8vLw2Wef4cMPP5QkY0hICMxm84Dn1NfXY/LkyeIEckNGoxHx8fHIyMiQOorbCgsLc7oIiIqKwtGjR2E0GhEZGYmLFy/i8OHDkuzru2rVKpSXl0Ov12PLli24c+cOAKCtrQ1GoxHbt2+H3W7HypUrRc92N51OB51O1+93Wq0WNTU1Iicikh92szKwm4eO3Xx/sJvdC/fdUJiKigrExsbizTff7LMKqY+PDzZu3IiYmBhJp7Q999xzqKysdJnh4MGDOHv2rNNqluRMEASEhoZKHcOtxcXFoa6uzjG97dVXX4UgCFi2bBliY2OxcOFC2Gw2rF27VvRskZGReO+999DY2IiVK1fiq6++AgA8/fTTWL9+PZqamvD+++9DrVaLno2I7h27WRnYzUPHbiYl4hN2henq6kJERMSA58ycORP19fUiJepr9erVOHz4MAwGAxYsWIC2tjYAwDfffIPTp0/j+PHjmDhxIl5++WXJMsrdjBkzcP78ealjuLWlS5di/vz5jimgU6dOxZ49e5D6Ph1JAAAIB0lEQVSbm4tr165h+vTpWL58udNepsOluLgYarXaqeSXLFmC6Oho5Ofnw2Qyob29Hb6+voiIiMCyZcsQFhY27Lnu9ttvvwEAHn74YXh4eDiO/42QkJDhikUke+xmZWA3Dx27+d6xm92fYB9o1QsacZYsWYJHH30Un376qctzNm7ciOvXr2P//v0iJnN248YNvP322469If9p5syZ2LFjB/+IDKChoQE6nQ7btm3DCy+8IHUcGiK1Wg2DwQCDwQAASEhIQHp6OpYvXy5xMmdqtRqCIODQoUMIDQ11HA9GEAScPXtWhIRE8sRuVgZ288jCbiax8Am7wqxevRoZGRlYvHgxZs+e3ef7kydP4ujRo9i5c6cE6f5n4sSJ+O6772A2m3H69Gm0t7fDz88PTz31FFet7EdWVlafz2JjY7Fp0yYUFBQgPDzcabuOXr1boJC8jR49Gn/99Zfj2GKx9LulktRSU1MhCAL8/PycjoloYOzmkYndPLKxm0ksfMKuMMXFxThy5AgqKiowe/ZsREVFOVairaurw6lTpzB37lwkJSX1+dnU1FQJEtO/8V/fhxIEAQ0NDfc5Dd1vCxYsgLe3N/Ly8hAUFAS1Wo3XX3+dF3REIwS7eWRiN49s7GYSCwfsCtM7DWawf/Z/3nmz2+2il4fVakVRURHq6+tx69Yt9PT09Jtx7969omWSs+rq6v/8szExMfcxCQ2HPXv24KOPPrrnO+KczkbkHtjNIxO7eWRjN5NYOCVeYbZv3y77aTAXLlyAXq9HW1vbgBcvcv89xMRiH9nS09MxduxYnDhxAs3NzaiqqkJISAgmTJggdTQner3+P/0cL/BJ6djNIxO7eWRjN5NY+ISdZOeVV17ByZMnsWbNGqSlpSE4OBgeHh5SxyKSjbsXupELV9M/XT057P2c0z+J5I/dTDQwdjMNFz5hV5hFixYhNjYWmzdvljqKS0ajEfHx8cjIyJA6CpEsGQwGaLVaqWP0YTabnY67u7uxYcMGXL58Ga+99hpiYmIc7+VWV1cjJycHYWFh+OKLLyRKTCQP7GYi98dupuEySuoAJK4rV67A09NT6hgDEgQBoaGhUscgki2DwYBZs2ZJHWNQu3btQkNDA4qKipCamoqQkBA8+OCDCAkJQWpqKgoLC/Hrr7/2u5IykZKwm4ncH7uZhgsH7AoTFhaGGzduSB1jQDNmzMD58+eljkFEQ1RSUoKkpKR+ty0CAH9/fyQlJaGkpETkZETywm4mIrGwm90PB+wK89JLL6GsrAznzp2TOopLmzZtQm1tLUpLS6WOQkRDcPPmzUHPEQThX51HNJKxm4lILOxm98NF5xSmpqYGX3/9NaqqqrB06VJoNBqoVKp+V3WValpPVlYWzpw5g4qKCkRHRyM8PLzfu4CCIHCvSyIZmzdvHrq7u/HDDz/Ax8enz/c2mw3JyckYPXo0jhw5IkFCInlgNxORWNjN7ocDdoW5e6/XgbZfkWplSFerWd6Nq1cSydvu3buRmZmJsLAwrFu3DlFRURg7dixaWlpQW1uLnJwcXL58GZs3b0Z6errUcYkkw24mIrGwm90PV4lXmPXr18t+j9Rvv/1W6ghEdB+kp6ejsbER+/fvx6ZNm/p8b7fbsWTJEl4QkOKxm4lILOxm98Mn7ERENKxqa2tx8OBBmM1m2Gw2+Pr64oknnkBqaqpbrKhLREQ00rCb3QcH7EREREREREQyxFXiiYiIiIiIiGSI77ArTO/CNoMRBAFnz54VIREREZGysZuJiMgVDtgVxtU7KTabDVeuXMGff/4JtVoNPz8/kZMREREpE7uZiIhc4Tvs5NDR0YHMzExUV1cjPz8fgYGBUkciIiJSNHYzEZGyccBOTux2OxYtWgSNRoMPPvhA6jhERESKx24mIlIuLjpHTgRBQExMDMrKyqSOQkRERGA3ExEpGQfs1IfNZsPt27eljkFERET/j91MRKRMHLCTk8rKShw6dAiPPfaY1FGIiIgI7GYiIiXjKvEKo9fr+/28p6cHv//+OywWCwBg3bp1YsYiIiJSLHYzERG5wkXnFEatVvf7uSAI8Pf3h0ajwYoVK/DMM8+InIyIiEiZ2M1EROQKB+xEREREREREMsQp8QQAaG9vR11dHby9vaHVajFqFJc3ICIikhK7mYiIOGBXmIKCAhQXFyM3NxdjxowBAJjNZqxatQqtra0AgIiICOzevRve3t5SRiUiIlIEdjMREbnCW7UKU1paip6eHscFAQBkZmbCarVi8eLFiIuLg8lkQkFBgYQpiYiIlIPdTERErnDArjBXr151WtymtbUVlZWVePHFF7Ft2zbk5uZi+vTpKCkpkTAlERGRcrCbiYjIFQ7YFcZqtSIoKMhxbDQaAQCJiYmOz6Kjox1byBAREdHwYjcTEZErHLArTEBAAKxWq+O4pqYGo0aNQmRkpNN53d3dYkcjIiJSJHYzERG5wkXnFGbKlCkoLy+H1WqFh4cHSktLodFo4Ovr6zjHYrFApVJJmJKIiEg52M1EROQKn7ArjF6vxx9//IG4uDjExcWhpaUFOp3O6RyTyYRp06ZJlJCIiEhZ2M1EROQKn7ArTGJiIrZu3YrCwkIAwMKFC5GSkuL4vqqqCp2dnZgzZ45UEYmIiBSF3UxERK4IdrvdLnUIIiIiIiIiInLGKfFEREREREREMsQBOxEREREREZEMccBOREREREREJEMcsBMRERERERHJEAfsRERERERERDL0f8xB97UcvKb2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x432 with 4 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "import seaborn as sn\n",
    "\n",
    "# because model tested on GPU, move prediction tensor to CPU then convert to array\n",
    "predicted_emotions = predicted_emotions.cpu().numpy()\n",
    "# use labels from test set\n",
    "emotions_groundtruth = y_test\n",
    "\n",
    "# build confusion matrix and normalized confusion matrix\n",
    "conf_matrix = confusion_matrix(emotions_groundtruth, predicted_emotions)\n",
    "conf_matrix_norm = confusion_matrix(emotions_groundtruth, predicted_emotions,normalize='true')\n",
    "\n",
    "# set labels for matrix axes from emotions\n",
    "emotion_names = [emotion for emotion in emotions_dict.values()]\n",
    "\n",
    "# make a confusion matrix with labels using a DataFrame\n",
    "confmatrix_df = pd.DataFrame(conf_matrix, index=emotion_names, columns=emotion_names)\n",
    "confmatrix_df_norm = pd.DataFrame(conf_matrix_norm, index=emotion_names, columns=emotion_names)\n",
    "\n",
    "# plot confusion matrices\n",
    "plt.figure(figsize=(16,6))\n",
    "sn.set(font_scale=1.8) # emotion label and title size\n",
    "plt.subplot(1,2,1)\n",
    "plt.title('Confusion Matrix')\n",
    "sn.heatmap(confmatrix_df, annot=True, annot_kws={\"size\": 18}) #annot_kws is value font\n",
    "plt.subplot(1,2,2)\n",
    "plt.title('Normalized Confusion Matrix')\n",
    "sn.heatmap(confmatrix_df_norm, annot=True, annot_kws={\"size\": 13}) #annot_kws is value font\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_7m8sgzHqVsy"
   },
   "source": [
    "**Unsurprising results - the model has trouble differentiating between 'neutral' and 'calm', and between 'disgust' and 'angry'.** \n",
    "\n",
    "If a human were asked to differentiate disgust and anger, getting 49/60 correct and 11/60 wrong wouldn't be too bad (these are the scores for 'disgust' from the above matrix). \n",
    "\n",
    "Other predictable results include confusion of 'sad' for 'disgust'. It's perhaps surprising that 'fearful' is confused for 'happy' as often as it is for 'sad' or 'disgust' - perhaps it is because fear is a truly multifaceted emotion. \n",
    "\n",
    "Based on this, I would compare in finer detail the features of confused emotions and see if there are any differences at all - and how to capture them. For real-world data, it would be much more productive to perform sentiment analysis on the spoken words translated to text, and consider that in our final evaluation. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xULpTS6h60EK"
   },
   "source": [
    "# Conclusion\n",
    "---------------\n",
    "Advances of the last 5 years involving upgrades to the autoencoder scheme have lead to the RNN, the upgraded LSTM-RNN, bidirectional LSTM-RNNs, and eventually to LSTM-RNNs with Attention layers to give profound temporal expressivity to the latent space of sequentially encoded data. The Transformer has built on this by taking advantage of parallelized self-attention layers to provide an almost truly global temporal representation of sequential data. \n",
    "\n",
    "Today, carefully thought out architecture building on these blocks leads to reasonable training times and excellent generalizability. We combine the CNN for spatial feature representation and the Transformer for temporal feature representation, and  augment the training data by increasing variation in the training dataset to reduce overfitting.\n",
    "\n",
    "CNNs are still the standard for encoding representations of spatial data. A CNN's filters' kernel sizes are important to both performance and accuracy, especially considering recent paradigms using smaller maxpool kernels such as those of the 3x3 strided 1 kernels in [VGGNet](https://arxiv.org/pdf/1409.1556.pdf), in contrast to the 11x11 stride 4 kernels as in the [AlexNet paper](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf).\n",
    "\n",
    "When we added convolutional and transformer layers beyond what was used here, that actually decreased test accuracy. Only as much complexity is warranted as is needed for generalizability. Although CNNs are good for images and transformers for sequential data, recently emerging paradigms (such as in this notebook) show that these networks are perfectly cross-applicable given careful thought. \n",
    "\n",
    "CNNs are powerful. Transformers work beautifully. They're even better together. Gone are the days of the LSTM-RNN.\n",
    "\n",
    "If you got this far, I sincerely appreciate your taking the time to do so. Feel free to drop me a line at ilzenkov@gmail.com with any feedback or questions you may have."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vSGFW0NY60EL"
   },
   "source": [
    "# References \n",
    "--------------\n",
    "- Ba et al, 2016. Layer Normalization. https://arxiv.org/abs/1607.06450\n",
    "- Bahdanau et al, 2015. https://arxiv.org/pdf/1409.0473.pdf\n",
    "- Cheng et al, 2016. Long Short-Term Memory-Networks for Machine Reading. https://arxiv.org/pdf/1601.06733.pdf\n",
    "- He et al, 2015. Deep Residual Learning for Image Recognition. https://arxiv.org/abs/1512.03385\n",
    "- Ioffe, Szegedy, 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. https://arxiv.org/abs/1502.03167\n",
    "- Krizhevsky et al, 2017. ImageNet Classification with Deep Convolutional Neural Networks. https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf\n",
    "- LeCunn et al, 1998. Gradient-Based Learning Applied to Document Recognition. http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf\n",
    "- Li et al, 2018. Visualizing the Loss Landscape of Neural Nets. https://papers.nips.cc/paper/7875-visualizing-the-loss-landscape-of-neural-nets.pdf\n",
    "- Masters and Luschi, 2018. Revisiting Small Batch Training for Deep Neural Networks. https://arxiv.org/abs/1804.07612 \n",
    "- Peng et al, 2017. Large Kernel Matters —— Improve Semantic Segmentation by Global Convolutional Network. https://arxiv.org/pdf/1703.02719.pdf\n",
    "- Santurkar et al, 2019. How Does Batch Normalization Help Optimization? https://arxiv.org/pdf/1805.11604.pdf\n",
    "- Simonyan and Zisserman, 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. https://arxiv.org/pdf/1409.1556.pdf\n",
    "- Srivastava et al, 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf\n",
    "- Ulyanov et al, 2017. Instance Normalization: The Missing Ingredient for Fast Stylization. https://arxiv.org/pdf/1607.08022.pdf\n",
    "- Vaswani et al, 2017. Attention Is All You Need. https://arxiv.org/abs/1706.03762\n",
    "- Wilson et al, 2017. The Marginal Value of Adaptive Gradient Methods in Machine Learning. https://arxiv.org/abs/1705.08292\n",
    "\n",
    "- Christopher Olah's Blog; Neural Networks, Types, and Functional Programming: http://colah.github.io/posts/2015-09-NN-Types-FP/\n",
    "- Lilian Weng's blog on Attention Mechanisms: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html\n",
    "- Stanford Autoencoder tutorial: http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/\n",
    "- Stanford CNN Tutorial: http://ufldl.stanford.edu/tutorial/supervised/FeatureExtractionUsingConvolution/\n",
    "- Stanford's CS231n: https://cs231n.github.io/convolutional-networks/\n",
    "- U of T CSC2515, Optimization: https://www.cs.toronto.edu/~hinton/csc2515/notes/lec6tutorial.pdf\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ChzuFbr160EL"
   },
   "source": [
    "# Appendix A - Convolutional Neural Nets Dissected \n",
    "--------------------------------------\n",
    "### [Back to main section: CNN Motivation](#CNN-Motivation)\n",
    "\n",
    "\n",
    "## Kernels and Filters\n",
    "\n",
    "At the heart of the behaviour of each convoluitional layer is its kernels. Each convolutional layer is made of a certain number of 2D kernels whose dimensions we specify .**A stack of 2D kernels in a layer create the 3D filters defining the operation of that layer on its input feature map (a tensor). Each filter contains a number of weights equal to its size (i.e. depth x height x width).** The kernel's size is a hyperparameter of the convolutional layer; the kernel's receptive field. It is so named because **the kernel's size determines the dimensions of the input feature map it sees during each convoluton.** \n",
    "\n",
    "**Even though complete CNN filters are 3D, they are only moved by their stride length in the H x W dimensions and encompass the entire depth of the feature map over which they operate - so the convolution is actually 2D, hence the nomenclature.**\n",
    "\n",
    "<img src=\"img/convolutional layer stanford.gif\" width=400 height=400 />\n",
    "<center><sub>Visualization of a single 3x3 stride 1 kernel's convolution with a 5x5 feautre map. Figure from Stanford Deep Learning Tutorial: http://ufldl.stanford.edu/tutorial/supervised/FeatureExtractionUsingConvolution/\n",
    "</sub></center>\n",
    "\n",
    "When performing a convolution, we take the products of the kernel's weights with the values of the input feature map and sum them to create a single entry in the output feature map. In this task, the initial input feature map has values describing pixel brightness - in our case, **we have 40x282 \"MFCC pixels\", each pixel representing the intensity in decibels of each of 40 MFC coefficients at each of 282 time points - in other words, we have a 40x282 grayscale image map with just one channel for brightness, which is why a CNN is a first-choice model for the task.**  Each kernel constituting a convolutional layer's filter creates a single 2D feature map - and so **more kernels = larger filters = more 2D feature maps = higher complexity (depth) 3D feature maps, where the # of kernels is specified by a layer's input channel.** If our input feature map has 1 channel and 2D dimensions H x W and we want 16 output channels, we'll have 1 2D kernel in that layer's filter and produce 16 2D feature maps of H x W, or a single 3D 16 x H x W feature map for the entire layer. **A higher output channel dimension creates a larger quantity of output feature maps. This may be beneficial or detrimental w.r.t. overfitting depending on the task.**\n",
    "\n",
    "For every input channel we have one 2D kernel, so when we have multiple input channels we have a collection of 2D kernels making up a 3D filter. Each 2D kernel creates a single 2D contribution to each output channel - so the entire 3D filter creates one entire 3D output channel. In this architecture, and as is standard for stacked CNNs, we expand the complexity of the input feature map in the channel/depth dimension while keeping constant its H x W dimensions. Each 3D filter is passed over the input feature map once for every complete output channel. The number of input channels determines the number of 2D kernels and thus, the *size of all 3D filters* in a layer, while the number of output channels determines *how many 3D filters* are passed over the input feature map - the convolution of all input feature maps with unique 3D filter creates one unique output channel. \n",
    "\n",
    "**To summarize, the input channel dimension determines the *size of all 3D filters* in that layer, while the output channel dimension determines the *number of unique 3D filters* in that layer. Each filter is defined by a unique set of weights, and each filter has its own bias term.**\n",
    "\n",
    "<img src=\"img/filter weights.png\" width=700 height=700 />\n",
    "<center><sub>A collection of 2D kernels makes up a conolutional layer's 3D filters: each filter has a unique set of weights, visualized here. The Kernel size is FxF and Depth is C (Channel dimension). Figure from Stanford's CS230: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks\n",
    "</sub></center>\n",
    "\n",
    "<img src=\"img/filter vis.png\" width=400 height=400>\n",
    "<center><sub>Visual of a 3-channel (RGB) 3D filter volume operating on a 3-channel 3D input volume to produce an output 3D volume - depth dependent on number of filters/output-channels of the conv layer.\n",
    "</sub></center>\n",
    "\n",
    "Each filter in a layer learns to produce a high-magnitude activation (after ReLU) when encountering a feature which it has learned to represent. Each filter outputs creates a unique feature map. In performant CNNs, the filters in each convolutional layer are likely to learn a different set of high-level features, in a hierarchical manner of increasing complexity with each subsequent convolutional layer as we expand the channel dimension. The idea behind this paradigm is to create higher-order feature representations akin to the way humans recognize images.\n",
    "\n",
    "Touching on the neuron representation of a CNN, **every element of a 3D output volume can be seen as the activation of a neuron that looks at a small region in the input volume.** Every neuron in a filter shares parameters with all neurons to the left and right of it spatially because the parameters are updated for these neurons by applying that same filter. **In this way, the neurons of a CNN are only locally connected in contrast to the fully connected \"dense\" layer.**\n",
    "\n",
    "**To be succint: A CNN layer transforms a 3D input volume to a 3D output volume of neuron activations. Classification itself is only done by subsequent non-convolutional layers, usually with fully connected linear layers.**\n",
    " \n",
    " \n",
    "## Zero Padding\n",
    "\n",
    "We zero pad the input feature map at the start of each convolutional block, i.e. add a column and row of zeroes to each side of the feature map's (H,W) dimension. **If we don't zero-pad before each convolutional layer then the input feature map is producing only 'valid' convolutions with the filters; after each full convolution the size of the output feature map would reduce by a small amount: The information at the borders will gradually erode.** If a kernel's next stride will set it to be out of bounds of the last column of the (H,W) dimensions of the feature map, that convolution would not be valid; the kernel will proceed (take a step of length==stride) down to the next row. If our kernel sizes is > 1 and stride length is > 1, this means we will completely lose features at the right and bottom edge of our feature map. Zero padding is standard in convolutional layers for kernel sizes > 1, even with a stride of 1 - if we don't pad in this case, corner pixels (features) will only be used once. **Because all convolutional kernels in this network are 3x3 stride 1; I zero-pad 1 to preserve the input dimensions of the feature map at each convolutional layer, i.e. 'same' zero-padding.**\n",
    "\n",
    "\n",
    "## Max Pooling\n",
    "\n",
    "This is where the real magic happens. Maxpool kernels operate on a feature map much like convolutional kernels; however, they perform no multiplication or addition, but rather a simple argmax operation taking the highest valued feature in the maxpool kernel's receptive field; In this way, maxpool layers reduce the dimensions of the feature maps (volumes) upon which they operate. Maxpooling can be equivalently seen as a dimensionality reduction, or down-sampling technique for features spaces. **Ultimately, maxpooling drastically reduces the numer of parameters a network must learn**; there are much fewer connections between subsequent layers than there would be without maxpooling. **Maxpooling is precisely what makes CNNs practical.** \n",
    "\n",
    "Intuitively, maxpooling outputs a feature map that contains only the most important features in each region of an input feature map, and puts these features in context of their position relative to the other most important features. Here's an extremely suitable analogy not far from what happens under the hood: **An eye near a nose near a mouth probably means a face - doesn't matter the position of the eye, it just matters that it's adjacent to the nose, and the nose to the mouth. Because of this paradigm, CNNs are extremely good at recognizing images even when rotated or shifted: that is, CNNs are feature and scale invariant. Note, however, that features themselves learned by CNNs are *not* scale or rotation invariant.** Again, and perhaps even moreso in this way, maxpooling is the absolute core of a CNN. \n",
    "\n",
    "<img src=\"img/maxpool gif.png\" width=500 height=500 />\n",
    "<center><sub>Visualization of a maxpool layer from Stanford's CS230: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks\n",
    "</sub></center>\n",
    "\n",
    "**To extend the above to this application: A sharp pitch transition from one mel frequency band to another could represent an angry voice, doesn't matter at what point in time the transition happens (shift invariance). This is precisely what we need to get the generalizability we want.**\n",
    "\n",
    "In this architecture I use non-overlapping maxpool kernels, reducing an input volume by the kernel size: **Using a 2x2 maxpool kernel takes 1/4 pixels in its receptive field, so it reduces the amount of information contained in its output volume by 75%.** The first layer has a 40x282 input volume = 11,280 pixels; after 2x2 maxpooling this becomes a 20x141 output volume = 2,820 pixels. 2,820/11,280 = 1/4. **While the dimensions of the volume are reduced the *dimensions* of the kernel, the total amount of information in the volume is reduced by the *size* of the kernel.**\n",
    "\n",
    "**The typical maxpool kernel on a first convolutional layer in stacked architecture such as AlexNet and GoogLeNet is 2x2 with stride 1.** A higher stride length throws away more information and a smaller stride length retains more, since they will take less or more steps over the input volume depending on their stride length, less so on their size. Some prominent networks (AlexNet, GoogLeNet) use 3x3 kernels with stride length 2 for their maxpooling layers. \n",
    "\n",
    "I optimized the kernel size and stride on the principal of finding a perfect balance between feature retention at each layer of the network and feature reduction to make the model computationally reasonable. **I found that a 2x2 stride 2 non-overlapping maxpool kernel - a 1/4 feature map reduction - was necessary on the input layer to retain a decent balance of features going through subsequent convolutional layers, but thereafter a non-overlapping 8x8, stride 8 kernel - a 1/16 feature map reduction - provided an excellent computation boost on the next 2 layers with little trade-off in accuracy.**\n",
    " \n",
    "**In practice, my final network architecture ended up to be that of VGGNet: 3x3 stride 1 convolutional kernels with 2x2 stride 2 maxpool kernels - with the distinction of 4x4 stride 4 maxpool kernels after the first convolutional layer out of computational necessity - I could not afford 138M parameters :).**\n",
    "\n",
    "\n",
    "## Regularization Using Dropout and a note on Pruning\n",
    "\n",
    "Each entry in a 3D activation map is akin to the activation of a neuron.  **When passing an activation map through a dropout layer, we simply deactivate a number of those neurons (activations); neurons to be dropped are selected from a Bernoulli distribution with probability _p_ to be zeroed (deactivated.** In this network we implement dropout of 30% of features at the end of each 2D convolutional layer after maxpooling the ReLU transformed activation maps. As in section 5.2 of [the dropout paper](https://arxiv.org/abs/1207.0580), the **weights learned in training must be scaled by a factor 1/(1-p) so that the weights are not inflated for those activations that are kept; this is necessary so the same network is performant without dropout on test data.** In essence, we keep the sum of the weights the same when using dropout, so we produce the same expected value of activations  independent of dropout probability - otherwise the activations from the kept neurons would cause incorrect weights to be learned. Dropout is applicable to a wide variety of neural net architectures. Think of it as kind of a \"weight loss\" for your network, pun intended - **making not only for faster training, but also reducing the co-dependence between learned features. Note that we turn off dropout (i.e. we keep all learned neurons) during validation and testing.** \n",
    "\n",
    "<img src=\"img/dropout.GIF\" width=400 height=400 />\n",
    "<center><sub>Dropout in the CNN, from https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf\n",
    "</sub></center>\n",
    "\n",
    "If we were to indeed turn off certain neurons during validation and training, that would no longer be considered dropout - that would be pruning, where we look for and remove all connections with \"dead\" neurons - those with little contribution or otherwise undesirable contributions to the network. When we perform pruning on a network, we would be removing certain clusters of neurons - feature representations, i.e. filters from a CNN. **There's been some recent progress in the field of CNN pruning with some pretty exciting results - for example, a [60% reduction in network parameters with a 0.6% accuracy loss](https://arxiv.org/abs/2001.08142).** I achieved very reasonable training times (~10 mins) so I spent more time on tuning than pruning.\n",
    "\n",
    "\n",
    "## Batch Normalization and Optimizing the Optimization Landsape \n",
    "\n",
    "The distribution of the activation map input to each convolutional layer change as the network is trained because the previous layer's output changes as its parameters are adjusted - the authors of [the batch normalization paper](https://arxiv.org/abs/1502.03167) term this \"internal covariate shift\". Non saturating nonlinearities (such as ReLU) are highly affected by weights of higher magnitude while saturating nonlinearities (sigmoid, tanh) may make a model slower to train. **Because the differential of a saturating nonlinear activation for outputs beyond +-1 approaches 0, the error differential between such outputs also approaches zero, meaning no change to backpropogation for such outputs - i.e. we encounter a variation of the vanishing gradient problem with saturing nonlinear activations.** Batch normalization re-centers the data, making it more likely that activation maps will belong to similar distributions throughout training. More specifically, **batch normalization encourages each layer's neurons to produce outputs closer to the linear regions of their activation**, i.e. attempts to center the activation.\n",
    "\n",
    "As activation map distributions change, the loss surface of the optimizer becomes more volatile; the loss surface is defined for  settings of the weights, which will be updated in unpredictable ways on unstable activation maps. As such, lower learning rates are usually needed to ensure weights are moved in the direction of a more stable negative gradient, otherwise an optimizer is more likely to converge on suboptimal solutions before proper model parameters are learned. **Though input feature maps may be initially scaled to have zero mean and unit variance, once they are output as activations of a subsequent hidden layer different features sets may transform to belong to different distributions.** Proper parameter initialization is also important to consider in avoiding suboptimal convergence.\n",
    "\n",
    "**Batch normalization ultimately results in faster convergence by steering a model away from becoming stuck in a saturated learning regime with a vanishing gradient.** Batch normalization also encourages regularized weights in this way, discouraging exploding gradients and overfitting. This normalization is implemented internally as an aspect of the model's architecture, so the output from each training minibatch is normalized at each layer. Since this mitigates the issue of internal covariate shift, **the network can be initialized with a broader range of parameters and a higher learning rate, and still achieve optimal convergenec.**  \n",
    "\n",
    "The paper shows BN works this empirically instead of with a mathematical proof - which is perhaps why some disagree with the reasoning behind BN: In [How Does Batch Normalization Help Optimization?,](https://arxiv.org/pdf/1805.11604.pdf) the authors argue that the stabilziation of the internal covariate shift is not the main factor behind BN's success, but rather **it is smoothing of the optimizer's loss function landscape that improves performance**. A smoother loss surface creates a more stable direction of the gradient differential between each training step i.e. moving towards a good minima; te gradients are more predictable leading to faster training since we can use higher learning rates when the gradient moves predictably in the direction of a global minima. We would otherwise need more frequent recalculation of the gradient in a region to make sure we move in the right direciton. \n",
    "\n",
    "**In short, the default behaviour of batch norm and as it is implemented in pytorch is essentially applying standard scaling (zero-mean, unit variance) to the tensor's minibatch (H,W) dimension.**  \n",
    "\n",
    "**Although originally BN was not used for validation and testing, as of 2019, an adapted version of BN is applied at both training and test times, improving performance: [Instance Normalization:\n",
    "The Missing Ingredient for Fast Stylization](https://arxiv.org/pdf/1607.08022.pdf)**\n",
    "\n",
    "\n",
    "## ReLU - Non-Saturated Activations are Healthier [for CNNs]\n",
    "\n",
    "We're using ReLU for activation, as is standard in CNNs to avoid saturation of weights - large weights aren't squashed so we  further discourage a vanishing gradient. **This model would take much longer to train with tanh/sigmoid with negligible performance differences.** This is especially important due to the moderate depth of this model, because with a saturating nonlinear activation we would have increasing difficulty trainig subsequent layers. We want a decent amount of layers to learn a higher representation of features, e.g. not just frequency transitions but the shape and magnitude of these transitions with respect to one another. The gradient is usually high if a neuron is activated with ReLU, producing effective successive gradients in the network, while sigmoid and tanh activations produce near-zero gradients for very high magnitude inputs. All in all, ReLU will lead to much faster convergence and for this task produces excellent results. \n",
    "\n",
    "\n",
    "\n",
    "## Turning Feature Maps into Probabilities with a Fully Connected Layer\n",
    "\n",
    "\n",
    "As noted, CNNs cannot perform classification themselves. Regular hidden layers operate only on 1D arrays, while convolutional layers produce 3D activation maps. As such, the entire volume of the 3D activation map is flattened to a 1D array so it can be connected to all input neurons in the next fully connected (FC) layer, thusly termed. Using a fully connected layer at the end of convolutional layers to enable classification is a standard architechture.\n",
    "\n",
    "**In this network, we have an output embedding from each convolutional block of 64x1x8 = 512, and an output embedding from the Transformer block of 1x40. As such, our final flattened embedding will contain 512x2 + 40 == 1064 activations, requiring 1064 input neurons in the fully connected linear layer.**\n",
    "\n",
    "The FC layer produces 8 activation logits based on the concatenated 1D activation map produced by the rest of the network. We need the FC layer in order to combine the high-level features learned by the end result of the convolution filters and the transformer encoder embeddings so that the network can build a global representation of the spectrogram. **While high level features may be present across different classes, a specific combination of these features is what confidently represents an emotion. Another way to see this is the FC layer evaluates which of the high order features learned by the convolution filters is present in the current sample.** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "21EQ903M60EM"
   },
   "source": [
    "# Appendix B - The Transformer \n",
    "****************************\n",
    "### [Back to main section: Transformer-Encoder Motivation](#Transformer-Encoder-Motivation) <br>\n",
    "\n",
    "## Self-Attention\n",
    "After the originalk Attention Mechanism was proposed by [Bahdanau et al in 2015](https://arxiv.org/pdf/1409.0473.pdf), Self Attention was first introduced in 2016 as intra-attention in [Long Short-Term Memory-Networks for Machine Reading](https://arxiv.org/pdf/1601.06733). We can learn all we need about self-attention from the Transformer paper. **Scroll past the wall of text to see a block-by-block breakdown of the Transformer with images.**\n",
    "\n",
    "The Transformer, Introduced in [Attention is All you Need](https://arxiv.org/abs/1706.03762) by Vaswani et all in 2017, the Transformer model replaces the seq2seq LSTM-RNN model with intra-attention described above by introducing the self-Attention mechanism as it is used now. **Instead of using LSTM-RNNs with attention layer-aligned context vectors, the Transformer implements multiple self-attention mechanisms and does away completely with LSMT-RNNs for seq2seq tasks.** Here's how:\n",
    "\n",
    "The Transformer architecture encodes an input sequence's context vectors as a set of (K, V) key-value pairs with the same dimension as the input sequence length. **Now, both the keys (inputs) and values (inputs' hidden states) comprise the encoder's hidden states. The output predicted at the previous timestep by the decoder is computed into a \"query\", and the next term in the decoder's output sequence is a mapping from the key-val pairs plus query: (Q,K,V)** Each output term of the decoder is a weighted sum of all values from the (K,V) encoded representation of an input - so, like a regular attention mechanism \n",
    "which decodes a weighted sum of hidden states, but self-attention assigns the (alignment) weights to each value (hidden state) as a sequence-length-scaled dot-product of the query with all the keys. **That is, the weighted sum of all the inputs' hidden states are computed w.r.t. the previous (last) term in the output sequence and the entire input sequence.** This is where the global attention ability of the Transformer originates. \n",
    "\n",
    "**An attempt at clarification: _EACH_ term in the final output sequence depends on _ALL_ terms in the input sequence _AND_ the previous term in the output sequence: Thus, _self_-attention.**\n",
    "\n",
    "<img src=\"img/self-attn-eqn.GIF\" width=300 height=300 />\n",
    "<center><sub>Equation behind the Self-Attention mechanism. From Vaswani et al, 2017. An extremely simple representation of an extremely effective idea. The scaled dot product is just scaling by dimension _n_ of source hidden states for sequence output at timestep _t_. \n",
    "</sub></center>\n",
    "\n",
    "And the result looks like this:\n",
    "\n",
    "<img src=\"img/self-attention.GIF\" width=300 height=300 />\n",
    "<center><sub>Visualization of the global temporal relationships captured by self-attention for text prediction. From Cheng et al, 2016.\n",
    "</sub></center>\n",
    "<img src=\"img/atten mechanism matrix.png\" width=300 height=300 />\n",
    "<center><sub>Another visualization of self-attention, this time for translation. From Bahdanau et al, 2015.\n",
    "</sub></center>\n",
    "\n",
    "\n",
    "## Multi-Head Self-Attention\n",
    "With self-attention in the bag, we can now apply multiple self-attention mechanisms: According to \"Attention is All you Need\", **the scaled dot product self-attention (Q, K, V) is computed over multiple \"representation subspaces\" - so each query, key and value has its own weight matrix associated. In this way, multi-headed (multi-layer) self-attention can compute a term in the output sequence weighted differently according to a region (subspace) of the input sequence.** Intuitively: In the sentence \"I like sour green \\_\\_\\_\", one attention head will learn weights more densely towards the term \"sour\" in the input sequence, and another will learn weights more densely towards \"green\". **Each attention head in multi-headed self-attention still computes a scaled dot product over the entire (K,V) encoded input, just weighted differently w.r.t. the input values.**\n",
    "\n",
    "The output of all the attention heads are concatenated and multiplied with a weight matrix which puts the dimension of the encoded state back to that of a single attention head; then **a single feedforward layer can operate on the encoded latent space regardless of the number of attention heads, and a softmax prediction is computed from a weighted sum of all layers in the multi-headed attention architecture.** The multiheaded attention layers are the meat of the transformer.\n",
    "\n",
    "<img src=\"img/multihead attention.GIF\" width=500 height=500 />\n",
    "<center><sub>Equation behind the Multi-head Self-Attention mechanism. From Vaswani et al, 2017. An almost-intuitive extension of the single-head self-attention mechanism.\n",
    "</sub></center>\n",
    "\n",
    "\n",
    "## Transformer Architecture\n",
    "\n",
    "<br>\n",
    "<center>The full architecture as implemented in 2017 in \"Attention is All You Need\":</center>\n",
    "\n",
    "\n",
    "<img src=\"img/transformer.GIF\" width=900 height=800 />\n",
    "<center><sub>From Vaswani et al, 2017. Each encoder/decoder block is stacked 6 times in the original implementation.\n",
    "</sub></center>\n",
    "<br>\n",
    "Explanation of each component:\n",
    "\n",
    "#### Input\n",
    "- The input embeddings are computed from the input sequence. \n",
    "- The output embeddings are given as the input embedding shifted one timestep (t+1) to the right.\n",
    "- The input embedding given to the transformer produces the (K,V) pairs while the output embedding provides the last predicted term, the query (Q). \n",
    "- The positional encoding in the diagram refers to a (sinusoidal) transformation of the embeddings that preserves relative positions of items in the input/output sequence embeddings. \n",
    "\n",
    "#### Encoder\n",
    "A a stack of 6 identical blocks, each of which is comprised of:\n",
    "- One multihead self-attention layer + a fully connected feedforward layer. \n",
    "- A residual (skip-connection) and a normalization layer after the multihead self-attention.\n",
    "- A residual connection and normalization later after the feedforward layer.\n",
    "- The residual connection adds the original embeddings with those output by the multihead self-attention layer.\n",
    "- The normalization layer is similar to batch normalization (BN), but adapted to sequential inputs and unlike BN, the norm layer is used at test time as well. The norm layer is applied to the combined embeddings from the residual connection.\n",
    "\n",
    "#### Decoder\n",
    "A stack of 6 identical blocks, each of which is comprised of:\n",
    "- Two multihead self-attention layers + one fully connected feedforward layer.\n",
    "- The output embedding given to the decoder is masked (the input embedding shifted to t+1) so we  do not pass any future terms from the input sequence (i.e t+2) to the multi-headed self-attention layers.  \n",
    "- A residual connection and normalization layer after each multihead self-attention layer and after the feedforward layer. \n",
    "\n",
    "#### Output\n",
    "- A final linear and softmax layer are added on to the final output of the decoder block. \n",
    "\n",
    "\n",
    "In sum total, this is the Transformer:\n",
    "- Does away with the optimization issues of the problematic gradents of the errors in the RNN by completely eschewing the RNN because it takes advantage of the attention mechanism.\n",
    "- Uses the attention mechanism to assignsa weight to each hidden state of an input sequence and produce a hidden-state-weighted output at each time step of the output sequence.\n",
    "- Uses the self-attention mechanism which uses the input sequence, all their encoded hidden states, and the previous (last) predicted word in the output sequence - (Q,K,V) - to produce an input-hidden-state-query-weighted output at each time step of the output sequence. \n",
    "- Ensembles self-attention mechanisms to create the multiheaded self-attention mechanism, capable of assigning multiple weights to each (Q, K, V) set of an input sequence \n",
    "- Produces an output which is the softmax of the weighted sum of multiple self-attention layers. \n",
    "- Bells and whistles like 6 stacked encoder/decoder blocks, layer normalization and residual connections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qjfrP4NN60EM"
   },
   "source": [
    "# Appendix C - From Autoencoders to LSTMs to Attention\n",
    "--------------------\n",
    "## Autoencoders\n",
    "\n",
    "**Understanding the autoencoder - the basis for RNNs and seq2seq models, which the Transformer replaces, will help get a better understanding of the motivation and operations behind the Transformer architecture.**\n",
    "\n",
    "Autoencoders essentially try to learn a function F(weight, bias, X) = X. It's trying to output a sequence similar to the one input. This becomes difficult when we give this task to a network constrained by a limited number of hidden neurons - that is,\n",
    "there isn't a single neuron in the encoder for every data point in X. Therefore, an encoder needs to learn a compressed representation of the inputs it is trained on. For example, Reconstructing a 100-pixel 10x10 image using 50 hidden neurons. \n",
    "For an autoencoder to be effective, there must be structure - correlation - between the points of the data which we wish to process with an autoencoder. The constraint recquiring an encoder to learn a compressed representation is also called the 'bottleneck'.\n",
    "\n",
    "<img src=\"img/autoencoder.png\" width=png height=650 />\n",
    "<center><sub>Standard/Vanilla Autoencoder, figure by Deepak Birla: https://medium.com/@birla.deepak26/autoencoders-76bb49ae6a8f</sub></center>\n",
    "\n",
    "Autoencoders are in fact quite similar to PCA which, in a way, encodes the structure of data it operates upon by computing eigenvectors of a covariance matrix such that projecting points along the principal component axes produces vectors encoding the greatest amount of variance (information) in fewer dimensions than available in the input data. In contrast to the autoencoder, the components produced by PCA are linear combinations of the original data. This has the advantage of producing fast and interpretable transformations, but the disadvantage of being incapable of encoding nonlinear relationships between features. **In this sense, an autoencoder is sort of an upgraded PCA - it too can perform dimensionality reuidction, but with the added ability to learn nonlinear representations of features - at the sacrifice of interpretability and computational efficiency.\n",
    "\n",
    "\n",
    "## The Sparse Autoencoder\n",
    "\n",
    "Sparse autoencoders extend the constraints of encoding beyond number of hidden neurons - we may choose more hidden neurons than input data points, instead imposing the sparsity constraint, adding an extra penalty term to our loss function. We parameterize the penalty term with a sparsity parameter close to 0 for ReLU and sigmoid activations,  close to -1 for tanh: i.e. inactive. I'll refer to 0 as an inactive neuron herein. **The objective of sparse autoencoding is to force the average activation of all hidden neurons to be lower than the sparsity parameter: most neurons in the hidden layer will not activate for any given set of features.**\n",
    "\n",
    "The sparsity constraint or, penalty term, whichever way we refer to it, is the Kullback-Leibler (KL) divergence: a function \n",
    "which computes the divergence between two Bernoulli random variables, i.e. in a distribution of 0 or 1. **A KL divergence penalty, the regularization term of an autoencoder, blows up when the average activation of hidden neurons in a particular layer are not near the penalty constraint _p_ - and so the average activation of the hidden layer is forced to be neither 0 nor 1, but close to _p=~0_, e.g. 0.05.**\n",
    "\n",
    "The decoder layers of sparse autoencoders are trained with a loss function  - the \"reconstruction term\" - which optimizes the precise reconstruction of an input, usually with mean square (MSE) loss. Ultimately, the result is that unless a neuron's activation on a particular set of input features seriously blows up the loss for the resulting reconstruction, then that neuron will be parameterized to reduce its activation to 0 for that input. **A sparse autoencoder's neuron will only learn a high activation on an input feature set if it is critical to accuracy of the model.** As such, a model benefits from sparse autoencoding in that each neuron of the hidden layer activates if, and only if it is given as input the precise feature which it has learned to represent. This has the inherent benefit of harshly penalizing overfit models. \n",
    "\n",
    "\n",
    "## The Variational Autoencoder, Decoders, and Latent Space\n",
    "\n",
    "The vanilla autoencoder computes a feature representation called the latent space, in analogy to the principal components of PCA. Once an input feature set is encoded into a latent space, several more hidden layers called **decoder layers have the objective to reconstruct the input feature set from the encoded latent space.** \n",
    "\n",
    "Since autoencoders seek to produce outputs similar to their inputs, autoencoders are a flavour of generative model - although **vanilla autoencoders cannot generate output they have never seen. Variational autoencoders (VAEs) are more appropriate to that task. In contrast to a single point in the latent space encoded by a vanilla autoencoder, VAEs instead compute a distribution of the latent space of their input.** We can then sample points from an input's latent space distribution to truly generate new output data.\n",
    "\n",
    "<img src=\"img/VAE.GIF\" width=650 height=650 />\n",
    "<center><sub>Variational Autoencoder, figure by Katsunori Ohnishi: http://katsunoriohnishi.github.io/</sub></center>\n",
    "\n",
    "Similar to the sparse autoencoder, we have the **KL divergence in the latent (encoding) layer(s),\n",
    "now constraining  the latent representation to be close to standard normal distributions, and the reconstruction term on decoding layers minimizing the distance (intuitively, information loss) between a reconstruction and an input, where the reconstruction is now computed by random sampling from an input's latent space distributon** - which is why we needed a normally distributed latent space in the first place.\n",
    "\n",
    "If we had not imposed a regularization term on a VAE's encoder layers, it would ultimately learn to minimize the decoder layer's reconstruction term by simply encoding distributions of near-zero variance and/or very different means - essentially, encoding points in the latent space - and the VAE degenerates to the vanilla autoencoder. **Enforcing a standard normally distributed latent space regularizes the mean and covariance matrix of distributions in the encoded latent space, avoiding overfitting to an input as the vanilla autoencoder is trained to do.**\n",
    "\n",
    "\n",
    "## The RNN, Seq2Seq, and Gradient Problems\n",
    "\n",
    "The encoder-decoder paradigm, in practice is implemented as recurrent neural networks (RNNs), commonly with long-short-term-memory (LSTM) cells to deal with the vanishing/exploding gradient problem - this is the modern squence-to-sequence (seq2seq) model. The seq2seq model usually employs an LSTM-RNN network to encode a variable length input sequence into a vector latent space representation, and an LSTM-RNN to decode that vector into a variable length output sequence. \n",
    "\n",
    "RNNs encode a hidden state for each input of a given sequence. A node (or neuron) in the RNN takes each input in the sequence, adds it to the hidden state computed from the input before it, and computes a new hidden state through an activation function such as tanh. This process is repeated for all time steps for a given sequence, so the hidden state of each time step is computed on the aggregated hidden states of inputs before it. Only the last neuron's aggregated hidden state is passed to the decoding network.\n",
    "\n",
    "<img src=\"img/rnn.jpg\" width=500 height=500 />\n",
    "<center><sub>The backprop through time constitutes the gradient problem with the RNN. Figure by Chengwei: https://www.dlology.com/blog/how-to-deal-with-vanishingexploding-gradients-in-keras/</sub></center>\n",
    "\n",
    "**The issue in this structure is that we can only calculate error for the final neuron's output - so each neurons' error depends on neurons at all time steps ahead of it. The of the final neuron must be backpropogated to all neurons in the same layer, not just to neurons in previous layers.** If the error of the final neuron is small (<1), it will become increasingly smaller and \"vanish\" as it is backpropogated multiplicatively at each time step. Similarly for a large error (>1), it will become increasingly large and \"explode\". **The RNN network will either not learn, or will take forever to learn - \n",
    "or not all of the weights will get updated.**  \n",
    "\n",
    "\n",
    "## The LSTM Cell \n",
    "One mechanism to alleviate the gradient problems of an RNN is the long short term memory (LSTM) cell. Intuitively, the LSTM\n",
    "cell is added to each node/neuron/time-step of an RNN and 'forgets' or 'remembers' hidden states - in essence, bypassing certain neurons in a sequence of time steps. An LSTM cell boils down to a combination of switch gates on each neuron - allowing a node in an RNN to remove or add information to the node, but often allowing information to flow through unchanged.\n",
    "<img src=\"img/LSTM.png\" width=400 height=400 />\n",
    "<center><sub>LSTM Cell from Christopher Olah's blog: https://colah.github.io/</sub></center>\n",
    "\n",
    "\n",
    "\n",
    "<img src=\"img/LSTM.jpg\" width=400 height=400 />\n",
    "<center><sub>Inner workings of an LSTM cell. Source: https://blog.floydhub.com/long-short-term-memory-from-zero-to-hero-with-pytorch/</sub></center>\n",
    "\n",
    "In the above figure, the input _x_ flows through 5 gates: 1 forget, 2 input, and 2 output gates. **Note that each gate has weights which are learned through backpropogation: Each LSTM cell learns its own weight matrix. Each LSTM cell takes as input C<sub>t-1</sub>, previous cell's long term memory, and H<sub>t-1</sub>, previous cell's short-term memory (classical RNN hidden state). Each LSTM cell outputs long-term memory C<sub>t</sub> and short-term memory H<sub>t</sub>.** \n",
    "\n",
    "- **1 forget gate (left, sigmoid): Decides which information from the long-term memory is discarded.** Takes product of long term memory from previous LSTM cell C<sub>t-1</sub> with the forget vector; The forget vector is generated when the current input x<sub>t</sub> and previous cell's hidden state H<sub>t-1</sub> (short-term memory) are added and passed through a sigmoid activation gate:  \n",
    "    - 1) f<sub>vector</sub> = σ(W<sub>forget</sub>⋅(H<sub>t−1</sub>,x<sub>t</sub>) + bias<sub>forget</sub>)\n",
    "    - C<sub>t</sub> = C<sub>t−1</sub>∗f<sub>forget</sub>+i<sub>input</sub>: Computes the new long-term memory C<sub>t</sub>.\n",
    "- **2 Input gates (center, sigmoid+tanh): Encodes new information into the LSTM cell's running long term memory.** The hidden state from the previous cell H<sub>t-1</sub> (short-term memory) and the input to the current cell x<sub>t</sub> are passed through two gates:  \n",
    "    - 1) i<sub>1</sub> = σ(W<sub>i1</sub>⋅(H<sub>t−1</sub>,x<sub>t</sub>) + bias<sub>i1</sub>): A sigmoid activation gate maps unimportant features to 0, discarding them, and computes i<sub>1</sub>: \n",
    "    - 2) i<sub>2</sub> = tanh(W<sub>i2</sub>⋅(H<sub>t−1</sub>,x<sub>t</sub>) + bias<sub>i2</sub>): A tanh activation gate operates on previous short-term memory H<sub>t-1</sub> and current input x<sub>t</sub> to compute i<sub>2</sub>.   \n",
    "    - i<sub>input</sub> = i<sub>1</sub>∗i<sub>2</sub>: The product i<sub>input</sub> is added to the LSTM cell's long-term memory C<sub>t</sub> by the output gate.  \n",
    "\n",
    "- **2 Output gates (right, sigmoid+tanh): Encodes the new short-term memory (hidden state).** \n",
    "    - 1) O<sub>1</sub> = σ(W<sub>output1</sub>⋅(H<sub>t−1</sub>,x<sub>t</sub>)+bias<sub>output1</sub>): Sigmoid gate takes hidden state from the previous cell H<sub>t-1</sub> (short-term memory) and the input to the current cell x<sub>t</sub>. \n",
    "    - 2) O<sub>2</sub> = tanh(W<sub>output2</sub>⋅C<sub>t</sub>+bias<sub>output2</sub>): tanh gate takes the new long term memory computed by this cell's input gate.\n",
    "    - H<sub>t</sub>,O<sub>t</sub> = O<sub>1</sub>∗O<sub>2</sub>: new short-term memory which is the current cell's output, is the product of O<sub>1</sub> and O<sub>2</sub>.  \n",
    "\n",
    "**The above LSTM cell architecture is added to all nodes/neurons in a traditional RNN. The above process is repeated in each and very LSTM cell. The long-term memory C<sub>t</sub> is a matrix running through all cells, while the short-term memory H<sub>t</sub> is the encoded hidden state output at each time step.**\n",
    "\n",
    "## Bidirectional RNN\n",
    "Yet another optimization on top of the LSTM-RNN is to make it bidirectional: we stack two RNNs together, where the input sequence flows forward through one RNN and backward through the other. Each node/neuron in the RNN contgains an LSTM cell. **At each time step the input is given simultaneously to two RNN neurons running opposite directions to eachother.** Each RNN aggregates its own hidden states, one forward and one in reverse. The hidden states are not shared between the two networks of a bidirectional RNN, but rather after the entire input sequence is consumed, the final hidden state of both the forward and reverse RNN is combined and then passed to the decoder network. \n",
    "\n",
    "<img src=\"img/bidirectional RNN colah.png\" width=500 height=500 />\n",
    "<center><sub>Bidirectional RNN from Christopher Olah's blog: https://colah.github.io/</sub></center>\n",
    "\n",
    "However, we still have a sequence of neurons to which we must backpropogate errors. Enter the Attention mechanism: \n",
    "\n",
    "\n",
    "## The Attention Mechanism\n",
    "Introduced by Bahdanau et al in 2015 in their paper [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473). **The attention mechanism introduces weights between all hidden states of an encoder RNN and the original input sequence using an attention layer, a simple feedforward network.** The attention layer uses \"alignment weights\" to compute the contribution of each input's hidden state to the final hidden state - also known as the context vector in the attention mechanism - which is then passed to the decoder network. For each term predicted by the decoder for the output sequence, the attention layer computes a probability distribution over all hidden states of the input. **In other words, each term in the output sequence is computed from a unique context vector from the distribution computed by the attention layer. Each context vector is an alignment-weighted sum of hidden states computed by an LSTM-RNN from an input sequence.**\n",
    "\n",
    "The attention layer is in contrast to a regular encoder-decoder RNN pair, which only uses one hidden state to generate the decoder's entire output sequence and just one hiddden state to backpropogate error. **The attention layer uses all hidden states to propogate error to all neurons of the encoder-decoder RNN simultaneously.**\n",
    "\n",
    "<img src=\"img/encoder-decoder-attention.png\" width=600 height=600 />\n",
    "<center><sub>From Bahdanau et al, 2015. The complete attention mechanism: Birectional RNN encoder-decoder pair with self-attention layer in between to compute a weighted context vector from all hidden states.\n",
    "</sub></center>\n",
    " \n",
    "In a sense, the attention mechanism is kind of like ResNet's skip connections: instead, we skip the contribution of neurons in an RNN layer. The skip, or shortcut connections in an attention layer are those between individual hidden states and the final context vector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Leyh5aMQ60EN"
   },
   "source": [
    "# Appendix D - Supplementary Notes\n",
    "------------------------------------\n",
    "\n",
    "## More CNN Kernel/Filter Math\n",
    "\n",
    "I'm going to refer to feature maps as an input/output volume interchangeably here. We concatenate the 2D volume produced by each kernel to create the 3D volume produced by the filter composed of those kernels. Let's ignore maxpooling to understand the impact of channel dimensions: In the first layer of this network, a 1x40x282 input volume becomes a 16x40x282 output volume which is fed into the next convolutional layer with a 32-kernel filter, producing a 32x40x282 output volume. The 3rd and final layer has a 64-kernel filter and produces a 64x40x282 output volume. Each kernel learns weights for the convolution it produces, so each filter is a set of depth x height x width weights - for example, the final layer contains 32-kernel filters, and it produces 64 such filters to satisfy its output volume of  64x40x282.\n",
    "\n",
    "The total number of 2D kernels in each layer = input_channel x output_channels. In this architecture, each convolutional block has 3 layers: First with 1x16=16 2D kernels, second with 16x32 = 512 2D kernels, 3rd with 32x64 = 2048 2D kernels. The size of all kernels in all layers in both convolutional blocks are 3x3 with stride 1 in this architecture. \n",
    "\n",
    "## Smoothing the Optimization Surface\n",
    "\n",
    "Apart from batch normalization, other methods for taming optimization surface include skip connections, where a layer's outputs skip some of its subsequent layers - so implemented by the extremely successful ResNet (residiual network) architechture - taking inspiration from certain cells in the cerebral cortex. Such skip connections require a weight matrix of their own to be effective if skipping over more than one nonlinear layer. Local minima worse than the global minima become shallower, i.e. their depth decreases and this eliminates \"bad\" local minima in the loss landsacpe. \n",
    "\n",
    "The loss landsacpe is formed by the loss function over the training samples and training weights. Due to the high parameterization of neural networks the visual representation of the loss landscape is not very intuitive without transformation. To this end, there are [specific techniques for loss landsacpe visualization](https://papers.nips.cc/paper/7875-visualizing-the-loss-landscape-of-neural-nets.pdf). \n",
    "\n",
    "<img src=\"img/optimization landscape.GIF\" width=800 height=800 />\n",
    "<center><sub>From Li et al, 2018. A visualization of an unruly (left) optimization landscape, and tamed (right).\n",
    "</sub></center>\n",
    "\n",
    "Smoothing the optimization surface: The depth of the loss function ultimately determines the degree of change weights will undergo due to the gradient. Deeper loss surfaces allows more flexibility in a model - for CNNs, that means the ability to learn higher level features built on outputs of previous layers, though but at the same time high-depth optimization landscape encourage overfitting to noise. In contrast, width smooths the optimization landscape and eliminates many bad local minima. We seek to find a balance in the width and depth of the optimization surface and ultimately make it as close to a convex function as possible."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Parallel_is_all_you_want.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "KERNEL",
   "language": "python",
   "name": "kernel"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
